ibm-granite/granite-3.2-8b-instruct

Granite-3.2-8B-Instruct is a 8-billion parameter 128K context length language model fine-tuned for reasoning and instruction-following capabilities.

462.2K runs
License

New model available: Try out Granite-3.3-8b-instruct View the new model

Playground API Examples README

Readme

Model Summary

Granite-3.2-8B-Instruct is an 8-billion-parameter, long-context AI model fine-tuned for thinking capabilities. Built on top of Granite-3.1-8B-Instruct, it has been trained using a mix of permissively licensed open-source datasets and internally generated synthetic data designed for reasoning tasks. The model allows controllability of its thinking capability, ensuring it is applied only when required.

Evaluation Results

<table>

<thead> <tr> <th style=”text-align:left; background-color: #001d6c; color: white;”>Models</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>ArenaHard</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>Alpaca-Eval-2</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>MMLU</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>PopQA</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>TruthfulQA</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>BigBenchHard</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>DROP</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>GSM8K</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>HumanEval</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>HumanEval+</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>IFEval</th> <th style=”text-align:center; background-color: #001d6c; color: white;”>AttaQ</th> </tr></thead> <tbody> <tr> <td style=”text-align:left; background-color: #DAE8FF; color: black;”>Llama-3.1-8B-Instruct</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>36.43</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>27.22</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>69.15</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>28.79</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>52.79</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>72.66</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>61.48</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>83.24</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>85.32</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>80.15</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>79.10</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>83.43</td> </tr>

<tr> <td style=”text-align:left; background-color: #DAE8FF; color: black;”>DeepSeek-R1-Distill-Llama-8B</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>17.17</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>21.85</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>45.80</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>13.25</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>47.43</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>65.71</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>44.46</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>72.18</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>67.54</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>62.91</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>66.50</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>42.87</td> </tr>

<tr> <td style=”text-align:left; background-color: #DAE8FF; color: black;”>Qwen-2.5-7B-Instruct</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>25.44</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>30.34</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>74.30</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>18.12</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>63.06</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>70.40</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>54.71</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>84.46</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>93.35</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>89.91</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>74.90</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>81.90</td> </tr>

<tr> <td style=”text-align:left; background-color: #DAE8FF; color: black;”>DeepSeek-R1-Distill-Qwen-7B</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>10.36</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>15.35</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>50.72</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>9.94</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>47.14</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>65.04</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>42.76</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>78.47</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>79.89</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>78.43</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>59.10</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>42.45</td> </tr>

<tr> <td style=”text-align:left; background-color: #DAE8FF; color: black;”>Granite-3.1-8B-Instruct</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>37.58</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>30.34</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>66.77</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>28.7</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>65.84</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>68.55</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>50.78</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>79.15</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>89.63</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>85.79</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>73.20</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>85.73</td> </tr>

<tr> <td style=”text-align:left; background-color: #DAE8FF; color: black;”>Granite-3.1-2B-Instruct</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>23.3</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>27.17</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>57.11</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>20.55</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>59.79</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>54.46</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>18.68</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>67.55</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>79.45</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>75.26</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>63.59</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>84.7</td> </tr>

<tr> <td style=”text-align:left; background-color: #DAE8FF; color: black;”>Granite-3.2-2B-Instruct</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>24.86</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>34.51</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>57.18</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>20.56</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>59.8</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>52.27</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>21.12</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>67.02</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>80.13</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>73.39</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>61.55</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>83.23</td> </tr>

<tr> <td style=”text-align:left; background-color: #DAE8FF; color: black;”><b>Granite-3.2-8B-Instruct</b></td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>55.25</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>61.19</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>66.79</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>28.04</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>66.92</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>64.77</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>50.95</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>81.65</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>89.35</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>85.72</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>74.31</td> <td style=”text-align:center; background-color: #DAE8FF; color: black;”>85.42</td>

</tr>

</tbody></table>

Supported Languages:

English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. However, users may finetune this Granite model for languages beyond these 12 languages.

Intended Use:

This model is designed to handle general instruction-following tasks and can be integrated into AI assistants across various domains, including business applications.

Capabilities

  • Thinking
  • Summarization
  • Text classification
  • Text extraction
  • Question-answering
  • Retrieval Augmented Generation (RAG)
  • Code related tasks
  • Function-calling tasks
  • Multilingual dialog use cases
  • Long-context tasks including long document/meeting summarization, long document QA, etc.

Training Data

Overall, our training data is largely comprised of two key sources: (1) publicly available datasets with permissive license, (2) internal synthetically generated data targeted to enhance reasoning capabilites. <!– A detailed attribution of datasets can be found in Granite 3.2 Technical Report (coming soon), and Accompanying Author List. –>

Infrastructure

We train Granite-3.2-8B-Instruct using IBM’s super computing cluster, Blue Vela, which is outfitted with NVIDIA H100 GPUs. This cluster provides a scalable and efficient infrastructure for training our models over thousands of GPUs.

Ethical Considerations and Limitations

Granite-3.2-8B-Instruct builds upon Granite-3.1-8B-Instruct, leveraging both permissively licensed open-source and select proprietary data for enhanced performance. Since it inherits its foundation from the previous model, all ethical considerations and limitations applicable to Granite-3.1-8B-Instruct remain relevant.

Model created