What an exciting time to be in Brighton those days! With Tamsen, it’s the first time we’re attending Reasons to previously known as Flash on the Beach!
On the menu: Carlos Ulloa, Mario Klingemann, Rob Bateman, Joshua Davis, Jared Tarbell, Stacey Mulcahy… yeah we mostly got there for code and we’re not disappointed at all!
Carlos Ulloa speaks about Music in VR, presenting his new project: Fantasynth. There was a really nice technical part about the engine behind the project, he chooses Unreal Engine over Unity due to the Sequencer editor. Also he brievly describes his own experience with Blueprints. That was hard to start with, but after months he found it particularly fine to work with! My own experience with Blueprints was really quick, so I didn’t explore it that much but still got the basics 😉 The strange thing is, he didn’t speak about binaural audio and HRTF filters directly. He did however mention important things like sound occlusion. Finally he spoke briefly of the.echonest.com/ acquired by Spotify and wearable haptic hardware found on Kickstarter so you can feel the sound through your body (like deep bass lines in a concert when all your body vibrates for example) .
Mario Klingemann introduced Convolutional Neural Network and t-SNE. Obviously he came back on Google’s DeepDream project and the concepts behind it. Then shared his own experiments with CNN like his own RasterFairy. He explained how thanks to t-SNE he classified a whole bunch of bitmaps depending criteria and how machines can learn by themselves. ArXiv.org was mentioned as a good resource for scientific papers showing off algorithms that would help process images. Basically, if you’ve heard or even tried the Prisma app, that was the idea. Not only can you apply a different style to your pictures like Prisma, but the next step is to actually have a neural network change the content. Imagine simply making something appear in an image, that wasn’t there, without photoshopping it in… And I’m not talking about making dogs appear everywhere like what everyone did with google deep dream and its default behaviour.
Another inspirational session was with Jared Tarbell, working on generative systems to create digital art or even tangible things such as his lifestar. A randomly generated cardboard igloo? I don’t really know how you can call it, go see the pictures. Each element of the structure is random but creates a beautiful structure you can walk in. It reminds me of good memories when I was playing with Biomorphs. The fun part is when you see the graphic paints itself thanks to code. It’s like watching someone painting. He used Processing for his demos, it seems to be a great tool for learning programming. This talk gives me many ideas for teaching programming to students which are first of all graphic designer.
Stacey Mulcahy works at Microsoft Garage. Her presentation went from AR & VR with WebVR and A-Frame to Fablab practice with Fusion 360 and AxiDraw to finish on Microsoft Bot Framework.
Rob Bateman‘s presentation was a case study on one of his games, and a great insight into what makes 3d faster than 2d basically. That’s not well put, but for example, a 2D game, heavy on vectors is going to render on the CPU (and be slow) while an Assasin’s creed mobile spin-off is going to run fine on mobile. A simple thing to realize is that 3d has a dedicated GPU that is NOT made for 2D. 2D could have gotten the same performance boost as 3D if people pushed for hardware accelerated 2D but only printers, through postscript, actually are hardware accelerated 2D (a chip in the printer that translates code to print 2D vectors and carries the load the CPU would carry instead). Sadly the world went for more 3d, he went on to show the disparity between CPU and GPU power… when your phone gets a powerful GPU, you don’t really need that much power with your cpu… That’s why as mobile developers, 2D will be slower cause you either have to raster 2D vectors and use them as textures on 3D geometry… or overlay 2D, cpu rendered, vectors, but at the cost of performance drop. Thankfully, Bateman has shown an interesting solution, perhaps the only one there will be : triangulating vectors so that we get 3D geometry and basically get the GPU to render vectors. So that means whatever is created in AnimateCC (even any kind of motion) needs to be exported as triangulated meshes. The demo we’ve been shown was amazing to look at, what were vectors that scaled up and down or were animated, were rendered effortlessly. Of course you don’t get perfect curves but that’s the next best thing. He created an AnimateCC extension to triangulate whatever’s on the timeline (Away Extensions) and creates a file with a custom binary file format containing all the geometry data and that’s read at runtime by AwayJS. That’s what he called tessellation (not to be confused with adding or removing triangles at runtime on 3D geometry in 3D games to change the level of detail).
To conclude, Joshua Davis was the last on stage, presenting his stuff Painting with Sound, Mesh and Shader. Like Jared, he uses Processing programming language and integrated development environment. He uses Fast Fourier Transform to use sound as data. Finally he played also with OpenCV to play around with his webcam output, to have it integrated into his sound visualisation workflow, or even have motion detection.
We left out a lot of speakers we went to see as their subject might not fit this blog, but each and everyone was awesome. Tamsen for example was especially pleased by Stefan Segmeister presentation which was centered around architecture and what’s beautiful and what’s not… and why functionality does not mean effectiveness if its not beautiful. So a lot of design stuff which we’re not necessarily directly concerned with at first, still had an impact on our way of thinking as whatever we do, we still are a part of a bigger team with a final objective in mind, and all members of that team are creatives. An app, or a game needs to be functional, we help to make that happen, but it also needs to be beautiful and we help make that happen too. Still has developers we are the ones with technical restrictions sadly but it’s our job to have as few as possible for the project we’re working on so that the vision of the project is carried out.
When the talks were over, back at the hotel Tamsen decided to download processing (with which he was toying with once he got reminded it existed at Jared Tarbell’s session) and decided to do a “logo remix” for reasons to to see what could be done in a short time with it. For a while he’s been wanting to use the golden angle somewhere, perfect opportunity to do so! So here it is! 
And if you want to become the next Joshua Davis – or just want to start doing something with sound, here’s a little start, with processing again. This is just a simple spectrum visualization. You need the minim library installed to make it run. It’ll get the first audio in it can find, if you can enable your “system mixer” audio as a recording device, you won’t have to plug an old mp3 player to your line in to get results.
import ddf.minim.*;
import ddf.minim.analysis.*;
float centerX;
float centerY;
Minim minim;
AudioInput in;
FFT fft;
void setup() {
frameRate( 60 );
noSmooth();
fullScreen();
centerX = width/2;
centerY = height/2;
background(0x000000);
minim = new Minim(this);
in = minim.getLineIn();
fft = new FFT(in.bufferSize(),in.sampleRate());
}
void draw() {
fft.forward(in.mix);
background(0);
int bands = fft.specSize();
float bandAngle = -TWO_PI / bands;
float oldLineX = centerX;
float oldLineY = centerY;
float minRadius = 80;
for(int i = 0; i < bands; i++) {
float bandVal = fft.getBand(i);
float r = bandVal * 20 + minRadius;
strokeWeight(5);
stroke(255, 255, 255, 255);
float lineX1 = r * cos(i * bandAngle) + centerX;
float lineY1 = r * sin(i* bandAngle) + centerY;
line(i== 0 ? centerX + minRadius : oldLineX,i== 0 ? centerY : oldLineY,lineX1,lineY1);
oldLineX = lineX1;
oldLineY = lineY1;
}
}
Reasons to is exactly the kind of event which give you a motivation boost of 1000%! Giving you the will to go even further and try new things. Creativity is just a matter of hard work and having fun!
