NeuralCrystal

The future is light.

A neural network etched in glass.
Light does the heavy math.

Reading handwritten digits

Most neural network tests start with the MNIST dataset. Here it is as a full optical neural network simulation, running in your browser.

Images become laser light

A micro-mirror array shapes pulses of laser light, and can stream patterns at high speed.

Light interacts inside the crystal

Small 1 µm features in the crystal layers bend the light and change its phase (shown in color). When rays with different phases meet in later layers, they produce interference patterns of light and dark seen by the imager.

Light points to the answer

Light lands on ten squares in the shape of a phone pad. The brightest square is the detected digit.

Your turn: draw a digit

Draw a digit on your screen. Your browser simulates the full neural network with light, in slow motion.

MNIST Results

The first goal was to match results from UCLA’s physical experiment, while miniaturizing the bench setup and using visible light. We used five trainable planes, as Lin did, and mirrored Mengu’s loss function.

97.3 %ACCURACY10 Digits • 10,000 Test Images
OpticalAccuracy
Random chance10.0 %
Lin 2018, UCLA✓91.8 %
Mengu 2020, UCLA✓97.2 %
NeuralCrystal✓97.3 %
MNIST datasetDigits crystal

Classifying photos

CIFAR-10 is the classic harder step after MNIST: small photos of ten everyday objects and animals, where non-linear computation pays off.

One crystal, multiple stages

Four stages allow for a deeper neural network, with a non-linear imager in-between. Predictions can be pipelined for parallel processing.

Non-linearity is essential for complex problems

Nonlinearity lets a model represent relationships a flat hyperplane cannot.

LINEAR

Hyperplane
Tilts, but cannot bend.

NON-LINEAR

A flexible surface follows
the peaks and valleys.

Imaging as a non-linear activation function

ReLU is a workhorse for training neural networks: it cuts off values below zero. Here, the imager’s readout provides that cutoff without a separate activation processor, providing strong non-linearity for training.

ReLU, leakyImager

OFF01011.53.0yLgamma 1.4saturated
L
a pixel's light ÷ the average (3 is 3× brighter)
y
what comes out: 0 is off, 1 is full

Stages in parallel

Each stage works on its own picture at once, so a new one can enter as soon as the first stage is free. We use that to read every photo twice: its mirror image follows one stage behind, for about 0.2 ms more, and lifts accuracy from 68.7 % to 70.2 %.

Try it

Pick a photo on your screen and watch the crystal name it.

CIFAR-10 Results

70.2 %ACCURACY10 Classes • 10,000 Hold-out Images
OpticalAccuracy
Random chance10.0 %
Li 2019, differential✓48.5 %
Rahman 2023, time-lapse✓62.0 %
NeuralCrystal✓70.2 %
ResNet-110, He 201693.6 %
ViT-H/14, Dosovitskiy 202199.5 %
CIFAR-10 datasetPhotos crystal

Playing checkers

Next, we taught the crystal to play a game. Show it a checkers board, and the crystal suggests the next move. With thinking on, it predicts your response too.

The crystal sees the board

We shine an image of the board into the crystal. The bottom left quadrant is the input board. The other quadrants hold the output of prior stages.

Thinking, in passes

Light passing through 4 distinct crystal stages decides each move. Predictions are pipelined to maximize use of the stages. When thinking is turned on, this is used to predict the opponent’s reaction to each move.

Moves and value, in light

The final pass lands on a move map: each cell pairs a starting square with a destination. Separate WIN, DRAW and LOSS regions judge the value of the move. Hover over a cell to read it.

You vs Crystal00

Drag or tap the black pieces to play.

Playing chess

Chess requires deeper thinking, and a crystal neural network with larger capacity. It plays with occasional mistakes and flashes of inspiration.

The crystal sees the board

We show the crystal the board as a two-color, 96 × 96 pixel image, doubled to 192 × 192. Each pixel is projected onto 16 mirrors (4 × 4), allowing for dithering in the hidden state of later stages.

A bigger crystal that thinks ahead

Chess trains a higher capacity crystal: 80 glass surfaces across 4 stages. With thinking on, we pipeline multiple boards to predict the opponent’s reply to the crystal’s top candidate moves.

A 64 × 64 move map

Every from-square and to-square pair is one cell on the sensor. The middle diagonal of identity moves is re-used for a win / loss / draw meter, capturing the model’s understanding of how well the game is going.

You vs Crystal00

Drag or tap the white pieces to play.

Scaling means new science.

To run a whole transformer, more of the work has to stay in light: a nonlinearity between layers, and multiplying two changing inputs for attention. Both are being worked on in labs today. Further investment in this research may yield large reductions in the power requirements for AI inference.

Q×Kᵀscalescoressoftmax → AA×Voutput

Recent papers

Tools used

Built with

Shellcaster

Learn more ↗

All design, code, agent fleet management and analysis was done from inside Shellcaster

Shellcaster is a Mac IDE made for working with agents: your terminal, your browser and your editor in one place. Mark up anything you can see, from a running site to a doc or a line of code, and your agents build it while you keep moving.

shellcaster.com
Training

RunPod

H100s for every crystal, rented by the hour

The code, the physics and the training runs

Early prototypes and optics analysis

AI model

Astra

3D schematic morphs and optics cross-check

Resources

Python package
Shellcaster
Run a crystal yourself
pip install neuralcrystal

Download a crystal below, then send it your own image: a digit, a photo, a checkers or chess position.

import numpy as np
from PIL import Image
import neuralcrystal as nc
from neuralcrystal import cifar10

crystal = nc.load("cifar10-crystal.safetensors")
photo = np.array(Image.open("frog.png").convert("RGB"))   # uint8 [32, 32, 3]
cls, scores = cifar10.classify(crystal, photo)
print(cifar10.LABELS[int(cls[0])])                        # frog
Crystals
Datasets
References