Naming Conventions
The usage examples use the following template variables:
Prepare Training Data
Tranining data is private, however the project bundles some public datasets for convinience. The following pretraining datasets are available:
Download Model Weights
Models are trained starting from base weights. Gradients from the updated models are then aggregated to update the base weights. To create and publish base weights follow Create a new model and Publish a new model.
Train Model
Models can be pretrained of finetuned, however the models bundled with the project repo currenlty support only pretraining.
Generate Text
To generate text with current model weights:
Create a New Model
New models are added to and shared through GoodWoolf.
Publish a New Model
Model code and configuration are shared through GoodWoolf. Model weights are shared through the Training Ledger repo.
Publish a Model Update
Publishing a model update is similar to publishing a new model. The difference is that the model weights are the result of pretraining, fine-tuning, or aggregation of model deltas, rather than the creation of randomized model weights.