maanantai 13. maaliskuuta 2023

Unity video playback on canvas

I toyed a bit with video playback on unity last  week, these are the findings, "how to play video on unity".

Setup

Put a video clip into Assets folder.

 

 Setup the video in inspector, transcode, resize etc. whatever you need.

Create a RenderTexture in Assets

 

Setup the render texture in inspector, I think size should match the video resolution.

GameObject

Create game object with Video Player component, that has VideoClip source set to video clip, Render Mode "Render Texture", and Target Texture set to Render Texture we created on the setup phase.

Create game object under Canvas object, add Raw Image component and set the Render Texture we created on setup phase, as the Texture.

 

Problems

at first I couldn't find any tutorials for Canvas video playback (I suck at internet search).

The tutorials I first found were doing Renderer override rendering, which was not my goal.

The first solution I had, had RenderTexture and a dedicated Material for UI which had render texture attached as color texture. Works, but this Raw Image approach is even simpler.

Links



sunnuntai 17. heinäkuuta 2022

ShaderToy bacteria

ShaderToy Bacteria2

How it works

I saw video of "ants" coding adventures ( Coding Adventure: Ant and Slime Simulations Youtube ) and I wanted to try to recreate something similar.


Something similar was agents / particles / bacteria, simulated with gpu on a pixel grid using shadertoy.

The bacteria has 2D coordinates, direction and "life left seconds" (packs nicely in 4 floats of RGBA framebuffer). 
The bacteria roams on screen  and leave behind a trail ("pheromones"). The bacteria sniffs 2 points ahead of itself (somewhat like antennas) for pheromone strength, which it then uses to turn towards the most powerful pheromone trail. For a little random "gains life" logic, the bacteria receives more life if it is on top of lots of pheromone.
Each bacteria has life, which expires at some point, once expired the bacteria spawns on random point on the screen.

Common

Common source contains settings / defines for the bacterial behavior. There are also all the structures and all the helper functions for the program. 

Buffer A

Buffer A contains the bacterial structure, each pixel can contain live bacterial data, there should be at least 2 in the FRAME_COUNT define, this way we have "previous frame" data in the -1 frame. The current "buffer" area is writable, and reading from same buffer same pixel where writing happens, is probably a bad idea.. but this does it anyways :o .. Hmm, yeh there should be a different buffer where to read the bacterial data, rather than this FRAME_COUNT hack, but, it seems to work :D

Anyhow, the previous bacterial data is read, "life logic" is run with the bacteria, and the new data is written into the current pixel. If the pixel corresponds out of current "buffer" discard happens. 

!TODO rethink the buffer logic, so that reading / writing doesnt happen on the same buffer/texture.

Buffer B

Buffer B contains the pheromone data for the bacteria, this is very very heavy shader.
First decay is applied to the pixel, pheromone decays over time.
Then for each pixel, all bacteria is iterated through, the pixel position is compared to a line between bacteria last and current position, if the pixel is close enough to that line, pheromone is added to the pixel.

Image

Final image generation just reads Buffer B data and applies transformation to that 1D pheromone data to produce beautiful color palette on screen, the algorithm is straight from https://iquilezles.org/articles/palettes .

Summary

The bacteria produces random images, that are somewhat pleasing, there is no deep patterns, as there is not enough logic or desire for the bacteria to interact with everything, or there is the 2 "antennae" sniffing logic and the "add life" logic, which account for the bacteria turning and staying alive longer, but to create more interesting patterns, there should be other goals too, like gathering resources and returning to base.

Another interesting thing would be to isolate these "life goals" for the bacteria in certain place, so that it could be easily replaced with other directives.




tiistai 9. helmikuuta 2021

What makes code maintainable?

What makes code maintainable?

an list of attempts to make code better. not an "end of all knowledge" list, but evolving and living with times.

I have thought these for a while and better to make a post of these than just ponder inside my homogenic head.

Clear (minimal) interfaces

Interface embodies an abstract requirement, that implementation fulfills. 
Minimality tries to ensure the coherency of the concepts/interface, as humans do have limited capacity to reason and understand concepts.

Clear concepts.

The definition of a concept is different for each person, depending on experiences and culture. I think the only way to make an abstraction/concept concrete and clear, is to make it clear to oneself by implementing, reading, testing, and by documenting.

Clear apis

Boundaries, is this a stand-alone api, or part of some other group, is it a whole, or a part. Is it independent of any other apis. Connections, usage, modules.

Self documenting naming

naming is one of the big 3 problems of software engineering, for example often great sounding naming scheme, but actually quite horrible is: 
  • everything is a "Manager"
  • naming variables according to what class they are "UIButtonTextObject uiButtonTextObject;" 

Consistent coding style

If each line of code has different style, programmer spends more time decoding code into mental models, codified coding style helps with read speeds and helps mitigate coding errors.

Tests

Tests of/on interfaces, Tests of whole (integration tests), unit tests.

Comments

Comments on weird/Complex code. For example if the code has had to have construct where the indexing starts from 0, but skips 1; 0,2,3,4,5,6 being valid numbers.

Documentation

Examples and usage examples.

CI Building

Continuous builds of the product and creating internal releases constantly, instead of panic mode building on release day.

keskiviikko 27. tammikuuta 2021

Mop; Tools/Tooling

this is a post, I will update, while proceeding and creating tools/tooling.

Tools

MOP uses flatbuffers for file-format serialization, fbs files define enumerations, structures, flags, almost everything. Usually Python and C++ sources are generated from the fbs files.

draw.io/diagrams.net is used to visualise the connections of the structures and files; to explain how everything forms coherent mesh/scene/animation/resource presentation.

Tooling

MOP has python scripts to generate/view mop files.

pyconvert is a collection of scripts to convert gltf files to mop binaries, or list the contents of mop binaries.
 

Sidenotes

  • git repo master should be used to compile flatbuffers flatc, the binaries provided by internets are old and incompatible, with some fbs definitions.
  • python has its quirks, but once it starts working, it seems usable.
  • fbs files needs to define if a structure can be a root structure.
  • fbs files can also define file endings, which is a bit curious thing.
  • flatbuffers is probably not meant for python, the usage patterns are really cumbersome. 

tiistai 26. tammikuuta 2021

Mop; Graphics/Game Asset format

Why?.. where I come from..

I have been creating game/graphics engines for years now, while doing that I need to import 2D/3D/Mesh/Scene/Text/Sound assets efficiently, without hassle.

With 3D objects/meshes, I first end up just generating simple primitives (Balls, Cubes, Planes) on the fly and for complicated things I use gltf or assimp

Assimp is full of features that you do not need and misses features that you need (while providing massive maintenance footprint), frankly it is ill suited for real-time applications. 

GLTF is modern, changing and quite universal format, except, when creating something new, one has to extend the format with "extra" data or creating own custom extensions, which leads to having to maintain custom exporters. I am also assuming that the performance is often bad due to having to support all possible permutations that GLTF file comes in (not all data is required by gltf specs, whereas my engine can require to always need 2 sets of texture coordinates, which is arbitrary, but acceptable requirement).

Custom 3D format gives a lots of benefits;
  • Defining structures that the graphics engine needs and nothing else.
  • Most of the error handling can be offloaded onto tooling side, assets are configured on tooling side to match the engine.
  • No dependencies on 3rd party, for example if one day I want to define meshes with bezier curves, I am free to do so. Or if the cubic-spline interpolation is mathematically imperfect, I can change the specification.
  • Optimization potential by the tooling (gltf, obj, etc. do not support texture compression right off the bat, often textures are just uncompressed).

Journey

2019 I was creating a vulkan based graphics engine and started designing 3D mesh/Scene format (kokkeli-mop) at the time I just sketched together some bare bone presentation of how a 3D mesh structures could be presented with flatbuffers. At the time I was trying to learn vulkan rtx raytracing and the 3d fileformat was just a side-thought of a side-project.

2020 I abandoned of trying to do graphics engine from scratch or rather from lowest level and opted to create graphics algorithms and engine side on top of another engine. During development, I considered using GLTF in engine but I reconsidered and revived my MOP fileformat for this project and after couple months of development, friend recommended writing a blog of the journey of developing MOP, hopefully documenting the reasoning behind design choices..

MOP So far..

As this is ongoing process, all things are in constant change, iterations.. once everything has been proven to work, I can say that MOP has reached status 1.0.

First iteration of drawing the MOP graph.. I can say this is purely academical work at this point.



Somewhere along the line, scene graph sketches, but still very academic work.

At the beginning I had defined all the attributes and material/descriptorset bindings, as uint32 ID's, but later on I decided that actually, everything should be bound by semantic strings, the mesh provides with semantics data that the graphics pipelines might use, in attributes, or in uniforms, or somewhere else. I reasoned that parsing a string and having logic that dynamically interprets data versus having hardcoded indexes, the dynamic way wins.

latest version as of jan. 27. 2021 .. Some improvements on naming, and red arrows determining where blobs of data separate into different files.

At the moment I have created fbs files that define flatbuffers structures. Python scripts that convert gltf files into MOP structures. Of the structures, I have tested that Mesh works, and I am currently working on the Material blobs. 

Next post is hopefully going to be a explanation of Mesh structure and if I have completed Material structure, also about that as well.

links:

perjantai 5. tammikuuta 2018

Modern C++

A friend is/was learning C++/opengl stuffs, and was annoyed with all the pointers in C++, I found this odd.. and well, when I threatened to message her in the evening, drunk, something about Modern C++, she suggested I should write a blog post about it.

So, Here I am, drunk, and trying to figure out what to write.

I asked about tips from, fellow coders at IRCNet #OpenGL (yes, I know C++, but I don't know everything about C++, its insane language, (and on the other fact, yes OpenGL is the most awsome graphics api there is)).
All the things here represents my point of view, if someone views the world differently, thats their problem/opportunity to correct my blasphemy.

I think the topics, or points of interests are as follows:
  • Tools
  • Coding conventions
  • collections (stl things, and string)
  • auto
  • enum & using & namespaces
  • nullptr
  • pointers & references, what why when how, and how god awful they really are.
  • lambda
  • initializer lists
  • threads
  • time
  • templates
  • exceptions
  • R-Value / L-Value semantics
  • move semantics
  • Boost libraries
  • (Hell no, I'm staying away from regexp)
  • debugging
  • links

Changes:

14.01.2018:
  • more about pointers
7.01.2018:
  • added more topics
  • update to std::string rant
  • enum & using & namespaces topic and content
  • content for "pointers", "auto", "initializer lists"
  • links added

Tools

So lets start at the tooling, currently it seems to be modern to approach C++ projects with CMake tooling, CMake can be used to generate project solutions for whatever toolchains you are using in the project, this includes xCode, Visual studio, and pretty much any sane IDE out there.

For source version control, git, mercurial.
For graphics debugging RenderDoc, nSight (nVidia). AMD also has its own tools GPU Perfstudio and the likes.
Python is nice to all sorts of scripting and maintanence, if you are doing bigger project, and need a build support scripting.
For IDE, Visual studio and Visual Studio Code, are excellent. Clion was also suggested as IDE, I personally have not used it, so can't really say anything about it.

Coding conventions

There are several coding conventions, mine, is a unique mix of BSD coding convention, and my own view that all code should concentrate on readability and simplicity, if there is a chance that code can cause bugs later on, due to how it was written, then the writing is wrong.
Anyhow, all code should go through a formatter ( http://clang.llvm.org/docs/ClangFormat.html ), choose one way to do everything, and let the machine churn the formatting to "One correct way to rule them all" code form (unfortunately the tool doesnt support BSD coding convertions, so I personally am in the minority). This eases up everything in a team, as all the code in source control are in one way.

Collections

STL-Collections (vector, unordered_set, unordered_map, list, string), are awesome, and everyone should learn them. There really is no reason to make C type allocations anywhere anymore, and these collections should be used to manage all sorts of memory allocations.


For example, allocation a buffer for an image, could be done:

int width = 100;

int height = 100;

int channels = 4; // 4 == RGBA

int bytesPerChannel = 1;

std::vector<uint8_t> buffer;

buffer.resize(width * height * channels * bytesPerChannel);
// and access the buffer with buffer.data();



std::string is also a collection. it is a very special collection, modern applications consider std::string to be utf8 coded strings.

Update: I had a constructive arguments about this on the ircnet, C++ strings are really old, and well storing utf8 in std::string and treating it as just a container (std::vector<char> style), it works, but doesn't really give tools for string parsing and stuff.

Fact is, that unicode parsing in C++ is shiet.. I deal with this with making a rule "std::string is utf8 encoded", and if there is a case where I need to get the rune representation/unicode codepoints, then Ill convert the string to full 32bit unicode points (funny fact about utf32, its fixed length!, once we get more runes that 32bit can store, utf32 is f*ckd).

Oh another thing, not a very important, but currently, Ill do all sorts of parsings with fmt libary, if I have a text of string, where I want to replace some part with another string, Ill use fmt to generete a string, like: auto str = fmt::format("Hey {0} this works", "cutie");

enum & using & namespaces

C++11 brought enum class, and using keywords, to help the sanity with namespaces in C++. C++ lacks modules or any sane kind of way to manage compilation units, source code etc. we just have a "raw" include thingy, that basically just copy pastes source code from one file into this file. The preprocessor does some smart things with defines and macros (not much really, it cant handle strings and comparisons, c preprocessor is poor mans programming language.. that some people take to bordering insanity.).
enum class foo; lets you define strongly typed enumerations, these are almost god sent, capsulating enumerations inside  the entity that defines the enumeration.
using keyword allows us to finally respect namespaces with aliases, previously all we had were typenames and #defines, now we have namespace respecting { using UTF8 = uint8_t; } kind of construct that seems readable.

auto

auto keyword was added in C++11, it allows you to forget, and write very maintainable code, by omitting the information about what kind of type is being handled. Auto lets the coder to ignore the type, and let the compiler make the decision of what the type is, using auto, as much as possible, lets coder to change types, around the code, much more dynamically (less refactoring), and often makes the code less verbose.
on the flipside, auto can make a codebase, daunting to read afterwards, as type is deducted at compile times, this can potentially make very simple code snippet, to be endless rabbit hole. (to find what the type really is, in the worst case, compile the code, and set a breakpoint there, and let the IDE tell you the type).

nullptr

nullptr is a C++11 addition, it allows the world to get rid of nonstandard NULL define. Often, if the codebase is riddled with NULL (or functions with void as empty argument in [int foo(void)]), it is a sign that a) codebase is old or b) whoever wrote it, hasn't stayed up to date with times.

pointers

Okay, C++11 brought lots of things to pointers, smart pointers, and nullptr type.

The shared_ptr is a reference counting pointer type, these pointers share a counter and a resource (the counter is thread safe, the resource is not), if the counter hits 0, the resource is deleted. weak_ptr is a partner for sharer_ptr, it is used to describe a non ownership relation to a resource, to use it, you have to convert it back to shared_ptr pointer.

unique_ptr owns the resource, once it gets killed, the unique_ptr deletes the resouce.

more often than pointers, references are used, everywhere possible, with containers, these things are immensely powerful.

void consume(char *data, size_t len) { ... }
char *content = malloc(1024);
readFile(myFile, content);
consume(content);
free(content);

// rather than do mallocs and frees or news and delete[]s, use raii
// and let vector do all that stuff:
void consume(const std::vector<char>& data) { ... } // we get length from vector

std::vector<char> content;
content.resize(1024);
readFile(myFile, content.data()); // not sure, might be actually &content[0]
consume(content);

with references, you could make assumption that references, cannot be nullptr (a sane assumption, I have not seen a codebase where this assumption has not been made).
also with references, it sort of can say to other programmers, what parameters are "in" and what are "out".. for example:

bool toInt(const std::string& str, int& value);

in that example, coder can see, that immutable string is the "input" parameter to the function, and "value" is out parameter. The example is a bit artificial in sense, that it returns bool, as indication did the transformation succeed (it is possible to code in this style), many coders, would prefer that function to return a int, and have it take bool& success, in. But this is entirely about what flavor you yourself prefer. Also in the example, if the codebase allows exceptions (historically, exceptions have been frowned upon, and if a library uses exceptions, it cannot be used on projects, that do not support RTTI or exceptions).
References, are easy to be thought of as "just pointers", but, unfortunately, they have some additionaly magic placed into them by C++, something that transforms non valua objects into real value objects, and some kind of magic called "&&" or "&&&&&" .. ( like auto&&&& foo; ).

Codebases, for example UI library or something, are totally possible to architect, without pointers any/every/where, with references and standard template library, by using std::map, std::list and std::vector (though, you have to know those containers, their allocation behavior, to a degree).

lambda

Lambda functions is the greatest thing since invention of bread and butter. They are bit complicated, with all the rules that comes with them in C++, its not like in C# or Java, where you can throw things around and just YOLO your way around. In C++ you can assign lambda functions to C function pointers (following certain rules, not always), and into std::function class (following certain rules). But once you understand the rules, lamdas lets you create all sorts of callback style programmings.

initializer lists

I'm not sure what should I say about these.. In C++ you can initialize stuff in constructors, using initializer lists:
asdasd
class Foo
{
private:
  int faa;
public:
  Foo() {}
};
/// options to initialize faa are
// a) in place (in header file, much preferred, as if you have multiple ctor, the
//    default value is set to 0 automatically)
int faa = 0; 
// b) initializer list in Foo ctor
Foo() : faa(0) {}
// c) default value in ctor parameter
Foo(int faa = 0) : faa(faa) {}

/// Also one other thing would be to use the new '{}' initializers
int faa{0};
Foo() : faa{0} {}
foo(int faa = 0) : faa{faa} {}

at the moment I am trying to start embracing the {} braket initializers everywhere to distinguish calls from function calls.

threads

C++11, brought threads, yey, finally. TODO

time

TODO

templates

TODO

exceptions

TODO

R-Value / L-Value semantics

TODO

move semantics

TODO

Boost libraries

Boost libraries seemed like a good idea at one time, when C++ did not see a release of specs every few years. At that time the problems usually were boost library incompatibility between versions, and different versions used by 3rd party libraries. I doubt this problem has gone away, I've lived my life as far from boost as possible, the code in boost libraries is usually deep template magic, and interconnections between different parts of the library. If you take boost in, your codebase will be married to it, for life. Boost contains solution for everything, polygons and maths? yes, build system? yes, reflection? probably, extended filesystem? yes, networking, yes.
I would avoid it at all cost.
Too much dependencies, with possibly deep dependencies to other parts of the framework. The alternative is to use many small libraries, and just get things done.

(Hell no, I'm staying away from regexp)

in life of a developer, regexp comes before you as a "good idea" every 1-2 years.. usually people deal with it, by relearning it each time. This is also the strategy I am using.
Regexp was added to C++XX, Im not sure, but maybe in C++11. I've used it 1 time, a month ago, on a libarary that was tossed out of the product, I am not suprised.

debugging

TODO

Links

  • cppreference.com pretty must up to date documentation on standard C/C++ (do not trust microsoft or any other specific vendors, they will screw you over).
  • github.com/fffaraz/awesome-cpp list of C++ libraries (not all, but good list of tools)
  • code.visualstudio.com "ide" for free from microsoft, requires a lot of tinkering to get it to compile whatever you want, cross platform!
  • visualstudio.com ide for "free" from microsoft, requires a little to get used to. windows only (dont be fooled by vs for mac, it is not visual studio).
  • cmake.org cmake tools for project generation (should be used to handle all library and stuff linking)

maanantai 25. marraskuuta 2013

requestAnimationFrame

Im alive.. I've been learning javascript (+golang) for few months. Maybe I'll blog about them soon.. but I just wanted to write something about requestAnimationFrame ..
I've been using timeout function in consistant manner in few projects ( http://icegem.net/flip/ , http://icegem.net/the-zombipeli-by-team-omnom/ , + some others ), at the moment I am learning & implementing some particle systems with WebGL ( http://icegem.net/webgl/ (maybe WebCL too in future)), when I got stuff "done" I proudly went and pasted url around the nets "Look another rotating triangle!".. at this time at #opengl ircnet it was pointed out that there was stuttering and the animations lagged.
To this I just thought "are they messing with me, this is just simple test that everything works".. wroong.. different suggestion came up "why there was stuttering" .. now after investigating, it seems that requestAnimationFrame was the culprit. It called the callback function usually at 16ms, but it also sometimes calls it after 32ms, in a pattern fashion. So animating things looked ugly.. On chrome canary this was not observed.

( the testing tool: http://icegem.net/webgl/req/ ) the canvas draws black dot at height of "how many ms passed.

on chrome (Versio 31.0.1650.57 m) I get the following (see those little dots at red X) those are calls that are late:


On chrome canary everything seems fine:

and the webgl animation:

Chrome (those little gaps in otherwise smooth grey gradiant):



Canary (no gaps (well those little gaps are from 16ms animation update, but! they are consistant):