paragekbote/gemma3-torchao-quant-sparse

A swift setup of gemma-3-4b with INT8 weight-only quantization and sparsity for efficient inference.

Public
68 runs
License
GitHub
Weights
Playground API Examples README Versions
  1. Author
    @paragekbote
    Version
    cuda12.1-python3.10-X64
    Commit
    b691813ef2e75369475c4639a4580ee2862800d2

    396049cb

    Latest
  2. Author
    @paragekbote
    Version
    cuda12.1-python3.10-X64
    Commit
    bb089c7886c920e0dd50e6d002d57b32cc0bbc98

    44626bdc

  3. Author
    @paragekbote
    Version
    cuda12.1-python3.10-X64
    Commit
    bb089c7886c920e0dd50e6d002d57b32cc0bbc98

    56167886

  4. Author
    @paragekbote
    Version
    cuda12.1-python3.10-X64
    Commit
    4fc4406854715fa734121ed81374dac58a0d0336

    761f5c47

  5. Author
    @paragekbote
    Version
    cuda12.1-python3.10-X64
    Commit
    8192e4347298cae48a9aeb5941ae4ab8e20b5438

    ee0d1f2a

  6. Author
    @paragekbote
    Version
    cuda12.1-python3.10-X64
    Commit
    9ecdb2634057d900c67930024429ee770d0396cc

    ff274c57