使用指南

“烧光”70000亿美元，与英伟达、台积电为敌, 梓豪谈芯, 腾讯科技 做财务预算确是一件得劲的事情。特别是可以毫不负责地规划一气，并且是百亿量级，后面跟的单位不是津巴布韦，而是美元。有种年轻时，假想中了彩票后如何规划使用，而兴奋一个晚上睡不着的赶脚。

Building Models That Learn From Themselves, Andrew Ng, Andrew’ Letters @ deeplearning.ai Data matters # Llama 3 的 pretraining 的数据量已经达到惊人的 15T tokens (1 token = 0.75 word)。从各种报道来看，现有的数据量和参数量都未能达到 Transformer 架构下的 LLM 所能达到的性能阈值。也就是说，更多的高质量数据还能不断提高现有 LLM 的能力。

LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report, Justin Zhao …, Predibase Abstract # First, we measure the quality of LLMs fine-tuned with quantized low rank adapters across 10 base models and 31 tasks for a total of 310 models. We find that 4-bit LoRA fine-tuned models outperform base models by 34 points and GPT-4 by 10 points on average. Second, we investigate the most effective base models for fine-tuning and assess the correlative and predictive capacities of task complexity heuristics in forecasting the outcomes of fine-tuning. Finally, we evaluate the latency and concurrency capabilities of LoRAX.

# LoRA Exchange (LoRAX): Serve 100s of Fine-Tuned LLMs for the Cost of 1 LoRAX 是一个用于降低基于同一基础模型（base model) 的多个 LoRA 微调模型的推理成本，并同时加速推理的 LLM 引擎。

Why We Need More Compute for Inference, Andrew Ng, Andrew’ Letters Much has been said about many companies’ desire for more compute (as well as data) to train larger foundation models. I think it’s under-appreciated that we have nowhere near enough compute available for inference on foundation models as well.

About, Adrian Colyer, The Morning Paper 最近又“一不小心”翻到 The Moring Paper，忽然想写点什么作为开始。