SciViz - A Web App for Gen AI Science Visuals
Following the journey I started in this post, I’m excited to announce that the project has now evolved into a working web app “SciViz”. This is an early demo that’s ready for anyone to try. It’s bare-bones for now, but it already demonstrates the technical foundation for generating engaging science visuals using AI.
Try the web app to the right
Just provide a link to a scientific text like
https://www.quantamagazine.org/what-can-birdsong-teach-us-about-human-language-20241121/
and hit generate.
SciViz takes a scientific article or text as input and generates a looping 2.5D video that encapsulates the core idea. While it’s not feature-rich at this stage, it opens the door to a new way of visually exploring scientific concepts.
Technical Workflow
Behind the scenes, SciViz operates through a carefully orchestrated system of APIs and workflows, integrating text processing, creative AI generation, and cloud-based video rendering to produce engaging visuals.
OpenAI API Integration: SciViz processes input articles or text through a workflow that combines my custom Griptape-based implementation with OpenAI’s API. Griptape handles intelligent web scraping to extract key details, while OpenAI helps refine these into a concise summary. This summary then serves as the foundation for generating a creative prompt.
Baseten Workflow: The model workflow, built and containerized (using Truss and Docker), is deployed on Baseten. My code processes the generated prompt and handles the entire video synthesis pipeline, demonstrating both the deployment of machine learning workflows and the integration with cloud infrastructure. This involves generating an image through a diffusion model, using another model to create a depth map and then using that depth map along with depthFlow for video synthesis.
Cloud Infrastructure: To ensure smooth operation and fast inference, SciViz leverages robust cloud resources:
GPU: 1 Nvidia A10G (24 GiB)
CPU: 8 vCPUs
RAM: 32 GiB
These components work together seamlessly to deliver results through a simple web interface.
My code for the project is available on GitHub: SciViz.
Limitations and Future Plans
As an early demo, SciViz is a work in progress:
Planned Enhancements:
Adding user options like camera movement styles.
Speeding up the generation process.
Enhancing the user interface for a more polished experience.
Fine-tune the aesthetic style.
Adjust the negative prompt to avoid aspects like adding text to the visual.
This web app represents the next step in exploring the potential of generative AI for scientific communication. SciViz is a personal project where I developed everything from the initial concept, along with the team at proem.ai, and custom code to model containerization (via Docker/Truss) and cloud deployment, integrating state-of-the-art AI workflows with scalable cloud practices to deliver a seamless user experience. It’s exciting to see how this can grow from a technical proof of concept into a usable tool. I look forward to hearing your thoughts, seeing what you create with SciViz, and sharing updates as it evolves!
Date: January 20th 2025