Releases: Lightning-AI/litgpt
Releases · Lightning-AI/litgpt
v0.4.4
What's Changed
- Tokenizer: fix issue when decoding a single token at a time by @Andrei-Aksionov in #1559
- Filter out false positive PyTorch scheduler warning by @rasbt in #1562
- Add an explanation on how to evaluate on a custom test set by @rasbt in #1564
- Remove duplicated bos_token for CodeLlama by @alealv in #1566
- Fix SFTDataset sample generation by @alealv in #1568
- Add Python API section to 0 to LitGPT docs by @rasbt in #1570
- Fix multi-turn prompting error handling and extra newline by @apaz-cli in #1569
- Add automatic downloading to CLI by @rasbt in #1571
- Bumb version for 0.4.4 release by @rasbt in #1572
New Contributors
Full Changelog: v0.4.3...v0.4.4
v0.4.3
What's Changed
- Fix Twine Build Issue by @rasbt in #1530
- Fix Mixtral prompt style by @janEbert in #1531
- Raise warning when loading a large model on a CPU device by @rasbt in #1532
- Tutorial
convert_from_litgpt
doc fix to removeoutput_dir
command argument by @jentfoo in #1533 - Test against eager attention on GPTNeoX by @rasbt in #1537
- Bump to 0.4.3.dev0 version by @rasbt in #1536
- Fix for LM Eval harness 0.4.3 by @rasbt in #1542
- Add
Phi-3-mini-4k-instruct
checkpoint by @rasbt in #1341 - Update phi-3 prompt template by @rasbt in #1544
- Add progress bar for weight loading by @rasbt in #1534
- Remove redundant code in checkpoint conversion script by @rasbt in #1547
- Update how evaluate tasks are listed by @rasbt in #1550
- Fix generate issue in finetuning script when finetuning with small datasets by @rasbt in #1551
- Tokenizer: check that
bos
andeos
tokens are applied w.r.t. to a provided config/args by @Andrei-Aksionov in #1527 - Add num_nodes argument by @janEbert in #1546
- Add Phi-3 Configs by @rasbt in #1553
- Update to version 0.4.3 for new release by @rasbt in #1554
New Contributors
Full Changelog: v0.4.2...v0.4.3
v0.4.2
What's Changed
- Disable "validating ..." comment if initial_validation is False by @rasbt in #1499
- Add simple issue templates by @rasbt in #1501
- Add note about dependencies by @rasbt in #1502
- Update test_readme.py by @Andrei-Aksionov in #1505
- Print message when unnecessary can't be deleted by @rasbt in #1507
- Bumb version by @rasbt in #1509
- LitGPT Python API v1 by @rasbt in #1463
- Fix pip upgrade command on windows by @rasbt in #1513
- add
LLM.generate
with streaming by @aniketmaurya in #1511 - Fix and update Python API docs by @rasbt in #1514
- Update LLM import and make stream/non-stream option consistent by @rasbt in #1515
- Extend Python API to initialize model with random weights by @rasbt in #1512
- Add Python API download option by @rasbt in #1516
- Add Rope group size as constexpr to fix thunder tests by @rasbt in #1520
- Explain how to access pretraining dataset options by @rasbt in #1510
- Bump thunder by @rasbt in #1519
- Add automatic checkpoint downloading in Python API by @rasbt in #1521
- Tokenizer: prefer HF Tokenizer by @Andrei-Aksionov in #1518
- iterate response from streaming server by @aniketmaurya in #1523
- Add new python API to readme by @williamFalcon in #1525
- Remove
init="local"
option by @rasbt in #1524 - Readme improvements by @rasbt in #1526
- Add
evaluate
to Readme by @rasbt in #1528
Full Changelog: v0.4.1...v0.4.2
v0.4.1
What's Changed
- Fix typos in Download Model Weights documentation by @rasbt in #1477
- 'Add models' docs by @rasbt in #1478
- Exclude prompt from generated response by @rasbt in #1485
- Make litgpt serve consistent with LitServe by @rasbt in #1466
- add multi turn prompt by @aniketmaurya in #1487
- Add docs explaining the streaming mode by @rasbt in #1484
- Fix Mistral 7B v0.3 weight download by @davmacario in #1486
- Fixes for latest lm_eval by @rasbt in #1489
- bugfix: LLM generated text with containing
{KEY}
would result in error by @aniketmaurya in #1490 - Add SAMBA to project gallery by @rasbt in #1493
- Add
resume="auto"
by @awaelchli in #1492 - Fix imports by @rasbt in #1491
- Pin NumPy version by @rasbt in #1495
- Version bump for new release to fix NumPy issue by @rasbt in #1496
Full Changelog: v0.4.0...v0.4.1
v0.4.0
What's Changed
- Set litdata < 0.2.6 by @carmocca in #1400
- Remove per-file CLIs by @carmocca in #1397
- Simillar -> Similar by @rasbt in #1405
- LitData: set
iterate_over_all
to False forCombinedStreamingDataset
by @Andrei-Aksionov in #1404 - Allow multiline prompts by @rasbt in #1279
- Explain dataset options by @rasbt in #1407
- Support
no_sync
with Thunder FSDP by @carmocca in #1414 - Minimal Python example by @rasbt in #1410
- Fix bug where LitData doesn't use seed by @bradfordlynch in #1425
- Add prompt style mapping for llama3 by @davmacario in #1406
- Simplify code by @rasbt in #1429
- OptimizerArgs by @rasbt in #1409
- Fix optimizer init with fused=True by @carmocca in #1434
- Fix learning rate calculation in pretrain by @rasbt in #1435
- Align readme by @rasbt in #1438
- Pin litdata by @rasbt in #1440
- Fix README.md alignment by @rasbt in #1439
- Update README.md for one last time by @rasbt in #1442
- A more centered look by @rasbt in #1449
- New CLI by @rasbt in #1437
- Update error message by @rasbt in #1453
- Explain how to list all available models by @rasbt in #1455
- Detect tensor cores by @rasbt in #1456
- Check checkpoint_dir and add
checkpoints
to path by @rasbt in #1454 - Add MicroLlama training support by @keeeeenw in #1457
- Streaming for serving with chat's generate function by @rasbt in #1426
- Fix sequence length bug by @rasbt in #1462
- Add
lr_warmup_steps
,max_steps
values validation by @shenxiangzhuang in #1460 - Fix issue where path in merge_lora is overwritten by @rasbt in #1465
- Option to skip expensive final validation by @rasbt in #1372
- Allow batch size "auto" setting in evaluate by @rasbt in #1469
- Warn users when there is a bnb mismatch by @rasbt in #1468
- Allow batch argument with batch recomputation by @rasbt in #1470
- LitGPT Python API draft by @rasbt in #1459
- Bump version for PyPI release by @rasbt in #1476
New Contributors
- @bradfordlynch made their first contribution in #1425
- @davmacario made their first contribution in #1406
- @keeeeenw made their first contribution in #1457
Full Changelog: v0.3.1...v0.4.0
Development release 0.4.0.dev0
What's Changed
- Streaming for serving with chat's generate function by @rasbt in #1426
- Add MicroLlama training support by @keeeeenw in #1457
- Check checkpoint_dir and add
checkpoints
to path by @rasbt in #1454 - Detect tensor cores by @rasbt in #1456
- Explain how to list all available models by @rasbt in #1455
- Update error message by @rasbt in #1453
- New CLI by @rasbt in #1437
- A more centered look by @rasbt in #1449
- Update README.md for one last time by @rasbt in #1442
- Fix README.md alignment by @rasbt in #1439
- Pin litdata by @rasbt in #1440
- Align readme by @rasbt in #1438
- Fix learning rate calculation in pretrain by @rasbt in #1435
- Fix optimizer init with fused=True by @carmocca in #1434
- OptimizerArgs by @rasbt in #1409
- Simplify code by @rasbt in #1429
- Add prompt style mapping for llama3 by @davmacario in #1406
- Fix bug where LitData doesn't use seed by @bradfordlynch in #1425
- Minimal Python example by @rasbt in #1410
- Support
no_sync
with Thunder FSDP by @carmocca in #1414 - Explain dataset options by @rasbt in #1407
- Allow multiline prompts by @rasbt in #1279
- LitData: set
iterate_over_all
to False forCombinedStreamingDataset
by @Andrei-Aksionov in #1404 - Simillar -> Similar by @rasbt in #1405
- Remove per-file CLIs by @carmocca in #1397
- Set litdata < 0.2.6 by @carmocca in #1400
Full Changelog: View on GitHub