Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
| Date | Stars |
|---|---|
| 2026-07-31 | 272 |
| 2026-08-06 | 272 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center"> <img src="https://github.com/ibm-granite/granite-3.0-language-models/blob/main/figures/granite-3_0-language-models-3x-v1.png" /> </p> <p align="center"> :books: <a href="https://github.com/ibm-granite/granite-3.0-language-models/blob/main/paper.pdf">Paper</a>  | :hugs: <a href="https://huggingface.co/collections/ibm-granite/granite-30-language-models-66fdb59bbb54785c3512114f">HuggingFace Collection</a>  | :speech_balloon: <a href="https://github.com/orgs/ibm-granite/discussions">Discussions Page</a>  <br> --- ## Introduction to Granite 3.0 Language Models Granite 3.0 language models are a new set of lightweight state-of-the-art, open foundation models that natively support multilinguality, coding, reasoning, and tool usage, including the potential to be run on constrained compute resources. All the models are publicly released under an Apache 2.0 license for both research and commercial use. The models' data curation and training procedure were designed for enterprise usage and customization in mind, with a process that evaluates datasets for governance, risk and compliance (GRC) criteria, in addition to IBM's standard data clearance process and document quality checks. Granite 3.0 includes 4 different models of varying sizes: - Dense Models: 2B and 8B parameter models, trained on 12 trillion tokens in total. - Mixture-of-Expert (MoE) Models: Sparse 1B and 3B MoE models, with 400M and 800M activated parameters respectively, trained on 10 trillion tokens in total. Accordingly, these options provide a range of models with different compute requirements to choose from, with appropriate trade-offs with their performance on downstream tasks. At each scale, we release a base model — checkpoints of models after pretraining, as well as instruct checkpoints — models finetuned for dialogue, instruction-following, helpfulness, and safety. ## Data Collection Granite 3.0 language models are trained using data from various sources such as unstructured natural language text and code data from the Web curated by IBM, a collection of synthetic datasets generated by IBM, and publicly available high-quality datasets with permissible licenses. For governance, all our data undergoes a data clearance process subject to technical, business, and governance review. This comprehensive process captures critical information about the data, including but not limited to their content description ownership, intended use, data classification, licensing information, usage restrictions, how the data will be acquired, as well as an assessment of sensitive information (i.e, personal information). For code, we annotate each code file with license information associated with the respective repository, found via Github APIs and only keep files with permissive licenses for model training. In addition, we also filter out all data obtained from sources that match URLs in IBM’s URLs blocking-list. Please refer to [Granite 3.0 Language Models technical report](https://github.com/ibm-granite/granite-3.0-language-models/blob/main/paper.pdf) for more details on the individual categories and datasets. ## Pre-training Granite 3.0 language models are trained on 10T to 12T tokens of language and code data, sourced from different domains. Data is tokenized via byte pair encoding (BPE, (Sennrich et al., 2015)), employing the same tokenizer as StarCoder (Li et al., 2023d). Below, we highlight the most important components of our pretraining strategy: **Data Mixture.** We craft the pretraining data mixture with two goals: 1) maximize the model’s performance across a diverse set of domains and tasks without bias toward a specific type of data or task; 2) leverage both high-quality and medium-quality data for optimal performance. To achieve these two goals, we adopt the 2-stage data mixture strategy used in MiniCPM (Hu et al., 2024) and JetMoE (Shen et al., 2024b). **Training Hyperparameters.** In Shen et al. (2024c), we proposed a syste
Excerpt of 10,361 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:a83486fdb9e1e809, llm:Repository name 'ibm-granite/granite-3.0-language-models' implies language models from IBM Granite 3.0; description/readme absent but project name indicates foundation language models.
matched fp:a83486fdb9e1e809, llm:Repository name 'ibm-granite/granite-3.0-language-models' implies language models from IBM Granite 3.0; description/readme absent but project name indicates foundation language models.
