Intermediate · 10 minutes

How to Run the Laya Decision Model Entirely in Your Browser with ONNX Runtime Web & WebGPU

Complete step-by-step developer tutorial to loading quantized 421M Laya weights into client-side browser memory via WebGPU and WebAssembly for sub-35ms typed decisions with zero server calls.

Step 1: Install ONNX Runtime Web and Hugging Face Transformers.js

Add onnxruntime-web to your frontend project. Ensure your build configuration allows serving WASM binary assets and WebGPU shaders.

npm install onnxruntime-web @huggingface/transformers

Step 2: Fetch Quantized Laya Model Weights (Int4 or Int8)

Download the quantized ONNX checkpoint from Vishal Mysore’s Hugging Face repository (VishalMysore/layaForWeb). The 4-bit model requires only 278MB of disk and browser cache space.

# Download the 4-bit quantized model for browser caching
curl -L -o public/models/laya-int4.onnx https://huggingface.co/VishalMysore/layaForWeb/resolve/main/laya-int4.onnx

Step 3: Initialize Browser Inference Session with WebGPU and WASM Fallback

Create an ONNX inference session specifying WebGPU as the primary execution provider and multithreaded WASM as the fallback for older hardware.

import * as ort from 'onnxruntime-web';

// Configure WebAssembly worker threads
ort.env.wasm.numThreads = 4;
ort.env.wasm.simd = true;

export async function createLayaSession() {
  const session = await ort.InferenceSession.create('/models/laya-int4.onnx', {
    executionProviders: ['webgpu', 'wasm'],
    graphOptimizationLevel: 'all'
  });
  return session;
}

Step 4: Tokenize Text and Execute Typed Decision Forward Pass

Feed the input text and candidate choices into the ModernBERT encoder. Softmax the output logits to obtain exact choice probabilities.

export async function classifySupportTicket(session: any, text: string) {
  const choices = ['billing', 'technical_issue', 'feature_request', 'security_breach'];
  
  // Format token inputs and run single forward pass
  const feeds = prepareLayaTensors(text, choices);
  const results = await session.run(feeds);
  
  // Compute softmax distribution
  const logits = Array.from(results.logits.data as Float32Array);
  const expLogits = logits.map(Math.exp);
  const sumExp = expLogits.reduce((a, b) => a + b, 0);
  const probabilities = expLogits.map(v => v / sumExp);

  const highestIndex = probabilities.indexOf(Math.max(...probabilities));
  return {
    selectedRoute: choices[highestIndex],
    confidence: probabilities[highestIndex],
    fullDistribution: Object.fromEntries(choices.map((c, i) => [c, probabilities[i]]))
  };
}