•1 min read•from Machine Learning
Pyrecall open source tool for detecting catastrophic forgetting during LLM fine-tuning[P]
Surprised there's no real tooling for this given how much research exists on continual learning.
Built pyrecall to fill the gap. Snapshots skill scores before/after fine-tuning, flags regressions, rolls back LoRA adapters by name.
Fully local, no external APIs. v0.1.0, MIT, pip install pyrecall
Curious if anyone has thoughts on the benchmark design that's the part I'm least confident about.
[link] [comments]
Want to read more?
Check out the full article on the original site
Tagged with
#rows.com
#natural language processing for spreadsheets
#generative AI for data analysis
#Excel alternatives for data analysis
#machine learning in spreadsheet applications
#real-time data collaboration
#real-time collaboration
#pyrecall
#catastrophic forgetting
#LLM fine-tuning
#continual learning
#LoRA adapters
#fine-tuning
#skill scores
#benchmark design
#regressions
#tooling
#open source
#roll back
#skill evaluation