Finetuner makes neural network fine-tuning easier and faster by streamlining the workflow and handling all the complexity and infrastructure requirements in the cloud. With Finetuner, you can easily enhance the performance of pre-trained models and make them production-ready without expensive hardware.
jina-aiThis release covers Finetuner version 0.7.0, including dependencies finetuner-api 0.4.10 and finetuner-core 0.12.3.
This release contains 4 new features, 4 refactorings, 4 bug fixes, and 4 documentation improvements.
🆕 Features
Allow Fine-Tuning for PointNet++ Models (#638)
We have added a new embedding model based on the PointNet++ model. You can use this model for 3D-mesh applications. To fine-tune it, set the model parameter of the fit function to pointnet++:
import finetuner
finetuner.login()
run = finetuner.fit(
model='pointnet++',
train_data='finetuner/modelnet40-train',
epochs=10,
batch_size=64,
learning_rate=5e-4,
loss='TripletMarginLoss',
device='cuda',
)We have also prepared a tutorial with detailed information about how to use the model and prepare 3D mesh data for Finetuner.
Add val_split to fit interface (#624)
To make it easier to evaluate models, we have added the val_split parameter to the fit function. Using this parameter automatically splits the training data into a training dataset and a validation dataset, used to calculate the validation loss. For example, the following call will automatically hold out 20% of the data for validation:
run = finetuner.fit(
model='efficientnet_b0',
train_data=train_data,
val_split=0.2
)Add recall and F1 score to evaluation metrics
If you are using the evaluation callback, evaluation results now include two new metrics: recall_at_k and f1_score_at_k.
Evaluation callback evaluates zero-shot models and reports this in the logs
Previously, the evaluation callback only evaluated models after each fine-tuning epoch. Now, evaluation metrics are also calculated before any fine-tuning and the log displays the metrics after each epoch, together with the pre-fine-tuning (i.e. zero-shot) evaluation results:
DEBUG Finetuning took 0 days, 0 hours 5 minutes and 39 seconds __main__.py:197
INFO Metric: 'pointnet++_precision_at_k' before fine-tuning: 0.56533 after fine-tuning: 0.81100 __main__.py:210
INFO Metric: 'pointnet++_recall_at_k' before fine-tuning: 0.15467 after fine-tuning: 0.24175 __main__.py:210
INFO Metric: 'pointnet++_f1_score_at_k' before fine-tuning: 0.23209 after fine-tuning: 0.34774 __main__.py:210
INFO Metric: 'pointnet++_hit_at_k' before fine-tuning: 0.95667 after fine-tuning: 0.95333 __main__.py:210
INFO Metric: 'pointnet++_average_precision' before fine-tuning: 0.71027 after fine-tuning: 0.85515 __main__.py:210
INFO Metric: 'pointnet++_reciprocal_rank' before fine-tuning: 0.79103 after fine-tuning: 0.89103 __main__.py:210
INFO Metric: 'pointnet++_dcg_at_k' before fine-tuning: 4.71826 after fine-tuning: 6.41999 ⚙ Refactoring
Drop support for Python version 3.7
Due to a new dependency in the stubs package of finetuner-core that is not supported by Python 3.7, Finetuner now requires Python 3.8 or higher.
Change default experiment_name from current working dir to default (#637)
Previously, Finetuner named experiments after the current folder, if no other name was provided. Now, the generic name default is used instead. Please note, that you can not create two runs with the same name in the same experiment, and we recommend always giving experiments explicit names.
Add page and size parameters to list_runs and list_experiments functions (#637)
We have added two new optional arguments to the list_runs and list_experiments functions: page and size. You can set size so that you retrieve no more than a specific number of runs or experiments. To retrieve more, set the page argument to progressively higher values. You can use these parameters to paginate the list of runs and experiments, which can become quite long.
Deprecate cpu parameter and notebook_login function (#631)
The cpu parameter of the fit function is deprecated. Instead, use the device parameter which can be set to cuda or cpu:
run = finetuner.fit(
model='efficientnet_b0',
train_data=train_data,
- cpu=False
+ device='cuda'
)Additionally, we have deprecated the notebook_login() function. This was necessary to specifically support Jupyter notebooks, but login() now works correctly in notebooks by itself. We have removed the notebook_login() function and you must update any code that calls it.
🐞 Bug Fixes
Fix build_encoding_dataset (#623)
Previously, when fine-tuning a CLIP model, if you passed Finetuner a list of strings to encode, it would fail because it was unable to determine if this was a list of texts or a list of URIs to images or other objects. You would need to explicitly state the type in a DocumentArray object. Finetuner can now automatically detect the data type and handle it correctly.
finetuner.encode(clip_text_model, ['some text to encode'])Adjust num_items_per_class if necessary
The finetuner.fit function has a num_items_per_class parameter which determines how many items per class should be put into a batch during training. Unfortunately, it is not possible to freely set this parameter for every batch_size and every dataset, which could lead to errors during the training. Finetuner will now automatically adjust num_items_per_class if the one provided by the user is not compatible with the rest of the configuration.
Finetuner will try to find a parameter value close to the one you provided. You will only receive a warning that the parameter has been adjusted, and training will continue.
Set default freeze to False
Until now, the build_model function would automatically set the parameter freeze=True when constructing a CNN model. Finetuner would also add a projection head to the function. To avoid this, freeze is now set to False by default.
Log messages in evaluation callback
Previously, some logging messages from the evaluation callback were overwritten by progress bars in some cases. This should no longer occur.
📗 Documentation Improvements
Rewrite the README (#638, #643)
We have rewritten the README file to be more concise and to include results from PointNet++ fine-tuning
Rewrite the M-CLIP notebook to use the German Fashion12k dataset (#643)
Our M-CLIP tutorial now uses the German Fashion12k dataset, which represents a more realistic application scenario.
Add before and after examples in the tutorials (#622)
In our tutorials, we now include some examples that let you compare results before and after fine-tuning.
Restructuring and enriching our documentation as a whole (#643)
We have performed a substantial re-write of our documentation pages. This includes new advanced topics like Negative Mining and more comprehensive information about inference in the Walkthrough section. We also improved the developer reference.
🤟 Contributors
We would like to thank all contributors to this release:
- Wang Bo (@bwanglzu)
- Michael Günther (@guenthermi)
- Louis Milliken (@LMMilliken)
- George Mastrapas (@gmastrapas)
- Scott Martens (@scott-martens)
