Training stack

What does PyTorch own in that stack?

The tensors, the autograd graph, and the kernels that run the step. Transformers, PEFT, and TRL call into that runtime.

When is TensorFlow still the framework you ship?

When the production graph, the SavedModel, and the serving path already are TensorFlow. New LLM training in 2026 usually starts in PyTorch.

What is Keras in 2026?

A high-level model API that can sit on TensorFlow, JAX, or PyTorch. Custom LLM training still drops to the framework underneath.

When is JAX the training framework?

When the lab wants NumPy-style code compiled and sharded by XLA across devices. The training step is a pure function, not a module full of in-place state.

Where does scikit-learn sit next to a neural net?

It fits the linear or tree baseline, the split, and the metrics. The neural net is compared with that baseline on the same split.

When is gradient boosting the model?

On tabular rows with mixed numeric and category columns, where a boosted tree usually trains faster and is easier to explain than a net.

How do XGBoost and LightGBM differ in the fit?

Both are histogram boosters. LightGBM grows leaf-wise, which can fit faster and overfit a small table sooner. You compare them on a locked validation fold.

What does CatBoost handle that the others make you encode?

Categorical columns. It builds ordered target statistics so a row's encoding is fit without that row's own label.

What is torchvision for beside an LLM stack?

Image datasets, transforms, and pretrained vision weights. An LLM fine-tune does not import it unless the model actually consumes pixels.

What does the transformers library load?

The architecture class, the config, and the weights for a named checkpoint, plus generate(). The tokenizer and the rows come from sibling libraries.

What must you record with from_pretrained?

The repo id and the revision, commit or tag. A floating name can resolve to different weights tomorrow.

What does the tokenizers library own?

The fast encode and decode, implemented in Rust and called from Python. Special tokens and the chat template have to be the same ones the serving process uses.

What does the datasets library store a split as?

Arrow tables with a named train, validation, and test split. A map() rewrite is cached, so a changed function must change the cache key or you will train on stale rows.

When do you stream a dataset instead of downloading it?

When the corpus is larger than the disk on the training node. You then cannot assume a random shuffle of the whole set without a buffer.

What does the Hugging Face Trainer hide?

The step loop, the accumulation, the log, and the checkpoint. You still own the collator, the label mask, and the metric that decides the best checkpoint.

What does Accelerate configure?

The device map, mixed precision, and gradient accumulation, and it launches one process per GPU. The model definition stays ordinary PyTorch.

What belongs in an Accelerate config rather than in code?

How many processes, which precision, and whether FSDP or DeepSpeed is on. Those change with the machine. The loss function stays in the training code.

What does the PEFT library attach?

A small trainable module, usually LoRA, on frozen base weights, and it saves only that module. The preference or SFT loss is computed by the trainer, not by PEFT.

When is a PEFT adapter the wrong artifact?

When you must change every layer's behavior and you have the memory for a full update, or when the serving stack you will use cannot load that adapter format.

What do you choose in a LoRA config?

The rank, the alpha, the dropout, and which projections get the adapter. Rank sets capacity. The target modules decide which parts of the block can change.

Which libraries does a QLoRA run use together?

bitsandbytes loads the frozen base in 4-bit, and PEFT trains a LoRA adapter in higher precision on top. Neither library alone is the whole method.

What does bitsandbytes provide in training?

An 8-bit optimizer and a 4-bit weight load so the base model fits. Those formats are for the training process. The server may want a different quantized file.

What does TRL run that PEFT does not?

The training objective: SFT on completions, or a preference loss such as DPO or GRPO. PEFT only says which weights move.

What rows does an SFT trainer in TRL expect?

Prompts and target completions, turned into tokens with the chat template, with the loss on the completion tokens.

What does a DPO trainer need that SFT does not?

For each prompt, one chosen completion and one rejected completion, plus a frozen reference model, usually the SFT checkpoint you started from.

What is GRPO in the 2026 TRL trainers?

A reinforcement step that scores a group of completions for one prompt. Reasoning runs often use a checkable reward, such as a unit test, instead of a learned reward model.

Why is the reference model part of the preference run?

The loss measures how far the policy moves from that reference. A different reference revision is a different experiment.

What does DeepSpeed change about a training step?

It shards optimizer state, gradients, and sometimes parameters across GPUs, and it can offload some of that state to CPU, under a JSON config.

What does ZeRO stage 1 shard?

The optimizer state. Weights and gradients still sit on each GPU. It cuts the Adam memory, which is often the first thing that does not fit.

What does ZeRO stage 3 shard?

Optimizer state, gradients, and the parameters themselves. A layer is gathered for the forward and backward, so the step spends more time on communication.

What is FSDP relative to DeepSpeed?

PyTorch's own fully sharded data parallel. It shards parameters, gradients, and optimizer state. You use it when you want that sharding without a second framework's config.

What is Megatron-Core for?

Training a transformer that needs tensor, pipeline, or expert parallelism, not only data parallelism. The model is split inside a layer, not only across data replicas.

When do you pick Megatron over ZeRO alone?

When one replica of the weights does not fit even after sharding the optimizer, so the layer itself has to be split across GPUs.

What does Lightning organize?

The training loop, callbacks, loggers, and device setup around a PyTorch module. The sharding still comes from FSDP, DeepSpeed, or the strategy you pass in.

What is torchtune?

PyTorch's own recipe set for LLM fine-tunes: configs for LoRA or full updates, the tokenizer, and the checkpoint, without the Hugging Face Trainer loop.

What does Unsloth change in a LoRA run?

It patches the attention and MLP path with kernels aimed at adapter training, so the step is faster and uses less memory. Confirm the saved adapter still loads in stock PEFT.

What does Ray Train schedule?

Copies of your training function across a cluster, with GPUs assigned to each worker and a place to write the checkpoint. The step itself is still PyTorch.

When is Ray Train the launcher instead of Accelerate?

When the run spans a cluster that must place workers, retry a dead node, and collect one checkpoint. Accelerate launches the processes you already have.

Who turns on mixed precision?

The launcher, Accelerate or Lightning or a DeepSpeed config, selects bf16 or fp16 for the compute. Master weights and the optimizer often stay in fp32.

What does gradient checkpointing trade?

Memory for compute. Activations are dropped and recomputed on the backward. PyTorch and transformers both expose the switch, and the objective stays the same.

What does torch.compile change about a step?

It traces the forward and fuses kernels. The first steps pay for the compile. Dynamic sequence lengths can trigger a new compile.

What is sequence packing in a fine-tune?

Short examples are concatenated up to the context length, with an attention mask that stops one example from seeing the next. Tokens per step go up.

Who applies the chat template?

The tokenizer, with the same template string in training and in the server. A trainer that concatenates roles by hand will not match generate().

What files does a LoRA run need to save?

The adapter weights and the adapter config, plus the base model id and revision. The base weights are not copied if the server will load them separately.

What should decide the best checkpoint?

A fixed evaluation set scored the way you will serve, not the training loss. Train loss falls when the model memorizes the fine-tune rows.

What does the run log besides the loss?

Learning rate, gradient norm, tokens per second, and the eval score. A tracker such as the one in the trainer is useless if those are missing.

What stays in Python when the step is in a CUDA library?

The config, the batch, and the launch. bitsandbytes, FlashAttention, and DeepSpeed run compiled kernels. A Python error at step time is often a mismatched kernel build.

What do you lock so the fine-tune can be rerun?

Package versions, the base revision, the data fingerprint, the seed, and the PEFT and trainer configs. A notebook that imported 'latest' cannot be rerun.

What is a practical 2026 stack when the model fits on a few GPUs?

PyTorch, transformers, datasets, PEFT, and TRL, launched with Accelerate. Add bitsandbytes for a 4-bit base, and FSDP or DeepSpeed when the full weights do not fit.