Session
Look who's talking!
Everything about media files in WASM: fonts, images, audios all in rust
Dmitriy Kovalenko - LightSource.ai
I made fframes. Fun fact: our fully featured video editor including media files loads faster than today’s SPAs. And we are doing A LOT of media processing: parsing font files for custom text shaping, decoding images, processing audios for visualization via fft, and many more.
All of this in WASM.
–
I’d like to talk about interesting things which I faced while doing my pretty unique media processing for fframes. Because it’s a framework for developers we are doing pretty unique job by statically in-lining assets which dramatically improves performance and load times. Besides that we are doing a lot of runtime specific media processing work like parsing font files and manually shaping unique glyphs inside a WASM for text shaping, doing FFTs over the audio files buffers in runtime with WASM in a very lightweight stream-based approach, using the generated in WASM audio buffers (by merging audio files or applying EQ for example) to play them directly with web audio api. And many more like colors parsing in compile time specifically for WASM, and working with images, etc.