Understanding Graphics

I thought that I was just going to give up on understanding how Libretro renders graphics. However, I still had this urge of solving this problem.

That turned out to be a good thing.

After reading on other engines and thinking just how Libretro handles this stuff directly, I took another look at the example code and actually built the example. I then spent my time reading the code, adding my own comments there. I found out that I was ignorant or mistaken in a few points.

  • A software framebuffer, in the case of Libretro, refers to a 2D array in memory. Each cell in that array represents a pixel.
  • The “cb” in “video_cb” means “callback.”
  • Retro_video_refresh_t, in reality, is a function that renders the actual frame, taking the data, both dimensions, and pitch.
  • Speaking of “pitch”, that actually refers to the number of bytes between any two lines in the buffer, that is, the length of a scanline. The “stride” variable refers to the pitch.
  • The mouse is not involved in the demo; I was looking at variables that had the name “mouse” in them. They actually refer to the pitch.

My plans have me uploading my comments on the code later. Right now, I shall give what seems to be the general procedure in software rendering using Libretro.

In a nutshell:

  1. Declare a bitwise 2D array in memory. This is an abstraction of your framebuffer.
  2. Call retro_video_refresh_t, putting its contents in a static variable.
  3. Call retro_get_system_av_info, putting information that relates to the framebuffer.
  4. Make a static void function.
  5. In that function, copy into a local variable the contents of the array in Step 1.
  6. In the same function, fill the cells you want in that array with pixel data.
  7. In the same function, put the buffer, its dimensions and pitch in the variable in Step 2.
  8. At runtime, call that function.
  9. At the time of actually loading the game, set the pixel format.

The issue now is that I have to figure out how to either convert lodepng’s RGBA format to RGB565 or how to use Libretro’s own rpng.

Either way, I do not feel lost anymore today.

Advertisements

The Framebuffer is the Way

Since my last post, I gotten some help at how to render graphics through Libretro. I mean, while I did take 2 classes in rendering graphics through GLUT (from OpenGL) and a bit of familiarity of Unity during my entire college career, I am still essentially new at graphics programming. I am just in a position to learn now.

I asked in the Libretro forums. Radius clarified to me that I am supposed to render to a framebuffer where I would set a pixel there to the color that I want. Physics engines can then be easily built on top of that concept. I then asked Radius about how I can go to learning about framebuffers in the first place. Meanwhile, given this point of clarity, I searched the libretro-samples repository for stuff on framebuffers. There, I found examples on how to use software rendering.

 

Basically, I would:

  • declare a bit-level variable that holds the framebuffer.
  • set the size of that framebuffer.
  • declare a static variable called “video_cb” that is of the retro_video_refresh_t type.
  • get the information of the AV system, including the dimensions, aspect ratio, and frames per second.
  • declare a couple of variables that hold the coordinates of the image I want to render.

I got difficulty going farther because, from reading the source code, the example is rendering a checkerboard. Apparently, the rendering itself involves copying the framebuffer to a separate “line” variable which holds an array, pushing the coordinates of the square to that “line” variable, then setting the value of those coordinates to a color, said color being a variable that holds a point in memory (0xff, specifically) to where you push the value while declaring that color variable.

(The example also apparently relies on the mouse. The mouse coordinates, which come from their own variables, use some “stride” variable that I do not know but is used in an intermediary buffer variable that copies itself to the “line” variable. There, the value of 0xff is put on the buffer variable, which is also an array, to some location that uses both some of its internal coordinates and that “stride” variable. I am not sure. I think I need to actually compile the source code if I were to understand.)

After all that, the example puts into the earlier “video_cb” variable:

  • the buffer variable
  • the specific dimensions
  • that “stride” variable that undergoes a left shift of 2 (whatever that means).

The above is inside a rendering the function that is executed at runtime.

At the time where the Libretro frontend actually loads the example, the example actually sets the retro pixel format. Afterwards, serialization makes a data array that is set to the image coordinates. A size of 2 is also involved in some way.

 

Changing our focus to the Libretro library itself, Libretro uses retro_framebuffer and retro_video_refresh_t. retro_framebuffer is a structure that has the following components:

  • the data that actually holds the framebuffer itself
  • the dimensions of the framebuffer
  • the “pitch”, that is, how many bytes take up the length of a scanline
  • the pixel format
  • access flags that say how “the core will access the memory in the framebuffer”
  • memory flags that tell the “core how the memory has been mapped”

I do not know about the existence, much less proper use, of these flags at the time I am writing this.

Except for the dimensions and access flags, most of the components use GET_CURRENT_SOFTWARE_FRAMEBUFFER, which returns a framebuffer directly to video memory. The thing is that GET_CURRENT_SOFTWARE_FRAMEBUFFER actually returns a pointer that must be writeable. However, actually rendering directly to the video memory instead of using a pixel format is optional. In this case, instead of directly setting the values of the buffer and “stride” variables, I would make a retro_framebuffer structure. In that structure I would:

  • set the dimensions normally.
  • set the access flags to RETRO_MEMORY_ACCESS_WRITE.
  • set the buffer variable to the framebuffer data.
  • set the “stride” variable to the framebuffer pitch that has a right shift 2 (whatever that means).

Of course, there is a fallback to the normal way.

retro_video_refresh_t is actually a type that Libretro defined, this type actually rendering the frame. The components of retro_video_refresh_t are:

  • data  from the above framebuffer
  • the dimensions of the framebuffer
  • a “pitch” which, according to the comments, is the number of bytes that make up the length between two lines in the framebuffer

 

While that is ample coverage of how do render to the framebuffer the data, there is still the issue on how to convert .png files to actual data in the first place. Fortunately, I looked up LodePNG. While that is not Libretro’s internal decoder of .png files, LodePNG has a simple way of decoding the file to pixels. I would simply declare a vector variable that carries the raw pixels, then I would also declare variables that carry the dimensions. I would then run LodePNG’s “decode” function that takes in the above 3 variables and the filename, putting the pixels in the vector variable. There, 4 bytes, representing RGBA in that order, represent each pixel. I am still not sure on how to use this raw .png data in Libretro, though.

 

I still am not sure on what is exactly a “framebuffer,” either.