Originally posted on Substack
Silence creates space for content to appear. -J.R. Rim
Video Art with FFMPEG
{{ youtube nobWeGycSe8 }}
One of the many tools we use to describe motion is in a sequence of images or videos. We love them and use them for our work as it adds yet another dimension to the images. The following few selections all use FFMPEG an open source tool to convert video from different formats in the command line or through a programming language. But there are a few more creative uses of this tool and Derrick Schultz has a great 4 week video course that he uses the tools in creative ways and provides code artists another way to create some interesting work.
If you are looking for a quicker fix, this presentation by Matic Lubej on ffmpeg is an overview on how to use FFMPEG specifically in Python using it creatively.
{{ youtube vivlxlxGkX4 }}
🖌️ Unconventional Media
Matrix-Webcam
This package displays your webcam video feed in the console.
Its a fun little tool to play around with; its open source so the code…
📸 Generative Graphics
{{ youtube 7He2MDAOI38 }}
Ken Perlin - How to Build a Holodeck
In the age of COVID-19 it is more clear than ever that there is a compelling need for better remote collaboration. Fortunately a number of technologies are starting to converge which will allow us to take such collaborations to a whole new level. Imagine that when you join an on-line meeting you are present with your entire body, and that you can see and hear other people as though you are all in the same room. There are many challenges to realizing this vision properly. The NYU Future Reality Lab and its collaborators are working on many of them. This talk will give an overview of many of the key areas of research, including how to guarantee universal accessibility, user privacy and rights management, low latency networking, design and construction of shared virtual worlds, correct rendering of spatial audio, biometric sensing, and a radical rethinking of user interface design.
Color Palette from Image
The following Observable notebook has an example of using the Vibrant.js library. It’s an interesting way to help choosing color palettes from a specific image.
🚤 AI / Deep Learning
https://imagen.research.google/video/videos/fairytale-2.mp4
IMAGEN VIDEO
We present Imagen Video, a text-conditional video generation system based on a cascade of video diffusion models. Given a text prompt, Imagen Video generates high definition videos using a base video generation model and a sequence of interleaved spatial and temporal video super-resolution models. We describe how we scale up the system as a high definition text-to-video model including design decisions such as the choice of fully-convolutional temporal and spatial super-resolution models at certain resolutions, and the choice of the v-parameterization of diffusion models. In addition, we confirm and transfer findings from previous work on diffusion-based image generation to the video generation setting. Finally, we apply progressive distillation to our video models with classifier-free guidance for fast, high quality sampling. We find Imagen Video not only capable of generating videos of high fidelity, but also having a high degree of controllability and world knowledge, including the ability to generate diverse videos and text animations in various artistic styles and with 3D object understanding.
We have seen alot of innovation in the image space when it comes to generative deep learning models. There is some interesting work in the audio generation / audio processing space as well. The following are a few more interesting work
Neutone
Neutone is a go-to platform for you to share real-time AI audio processing models with potential users in the audio production community.
{{ youtube KmB8z2CYjZY }}
Harmonai AI, Diffusion and Audio Generation Revolution
Listen to Harmonai core team’s vision and how their Dance Diffusion model is the first step on the journey to high quality music and audio generation for everyone.
đź”– Articles and Tutorials
{{ youtube A1Ud_liFn3U }}
Joshua Davis - Dribble Conference 2019
Joshua Davis is a design polymath—designer, technologist, author & artist who has used creative coding to generate algorithmic images and animations since 1995.
Josh Davis is one of the most prolific artist / designers and so much of his work provides a historical context on the progression of early generative works from 1995 to 2001.
đź“šBooks
Real-Time 3D Graphics with WebGL2
This book presents a clear roadmap to learning real-time 3D computer graphics with WebGL 2. Each chapter starts with a summary of the learning goals for the chapter, followed by a detailed description of each topic. The book offers example-rich, up-to-date introductions to a wide range of essential 3D computer graphics topics, including rendering, colors, textures, transformations, framebuffers, lights, surfaces, blending, geometry construction, advanced techniques, and more. With each chapter, you will “level up” your 3D graphics programming skills. This book will become your trustworthy companion in developing highly interactive 3D web applications with WebGL and JavaScript.
Tools
ct.js: 2D Game Editor
ct.js makes learning programming fun and game development easy by its visual tools, good docs and flexible, modular library. It is free, open-source, and is loved by hobbyists, professionals, teachers, and their students.