LLM
- 2401.02412 LLM Augmented LLMs: Expanding Capabilities through Composition
- Related Reddit: Expanding Capabilities through Composition (CALM) : r/LocalLLaMA
- 2312.15166 SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling
- GitHub - pratyushasharma/laser: The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction
- mixtral: 2401.04088\ Mixtral of Experts
- Phi model is free ! microsoft/phi-2 · Hugging Face
- LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
- GitHub - pratyushasharma/laser: The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction
- Paper page - LLaMA Beyond English: An Empirical Study on Language Capability Transfer
- GitHub - microsoft/TransformerCompression: For releasing code related to compression methods for transformers, accompanying our publications
- Evolving New Foundation Models: Unleashing the Power of Automating Model Development
A.1) QA
A.2) MoE
- Paper page - Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM
- Paper page - Soaring from 4K to 400K: Extending LLM’s Context with Activation Beacon
- Mixture of Experts Explained
- mlabonne/phixtral-4x2_8 · Hugging Face
- Mixture of Experts for Clowns (at a Circus)
B) RAG
- 대규모 언어 모델을 위한 검색-증강 생성(RAG) 기술 현황 - 2/2편 - 읽을거리&정보공유 - 파이토치 한국 사용자 모임
- Long-Context Retrieval Models with Monarch Mixer · Hazy Research
- RAFT: RAG 기법을 활용한 LLM 검색 증강형 미세조정(RAG + FineTuning) - 읽을거리&정보공유 - 파이토치 한국 사용자 모임