[{"content":"Disclaimer, this guide was done for my latest involvement with a React app, thus the examples are based on React 17 and thus this article will not address async rendering, suspense, etc. Also, the docs linked are from the legacy documentation for React 17.\nProblem Statement React, despite being one of the most popular front-end libraries, doesn\u0026rsquo;t have primitives, nor in the framework, nor in the underlying language to ward off performance issues. This is why it\u0026rsquo;s important to understand how the React ecosystem works and how to optimize it.\nDOM reconciliation optimization React is a declarative library that allows you to describe the UI as a function of the state. The desired state of the UI is described in the render method of a component in a virtual DOM, an in memory representation of the desired state. React will handle reconciliation to update the real DOM with the changes when deemed necessary.\nWhat is important to consider when writing components isn\u0026rsquo;t when React will update the DOM, this is a solved problem by the React team, but how costly the reconciliation process can be computationally expensive if one is careless:\nReconciliation of elements of different types - avoiding changing the type of elements For elements of different types, say a div and a span, React will unmount the old element and mount the new one. This will lead to a full rebuild of the subtree of the element, which can be costly if the subtree is large. One should avoid changing the type of element if possible at the top of a component tree.\n// Before \u0026lt;div\u0026gt; \u0026lt;Counter /\u0026gt; \u0026lt;/div\u0026gt; // After \u0026lt;span\u0026gt; \u0026lt;Counter /\u0026gt; \u0026lt;/span\u0026gt; The example above will destroy and re-create the Counter component.\nReconciliation of elements of same type - using the memo hook For elements of the same type, React will look for differences in the element/component\u0026rsquo;s props then only update in the DOM what has changed, then do the same recursively for the element\u0026rsquo;s children. To avoid performance bottlenecks when comparing object props, React compares object props by reference and primitives by value!.\n// Before \u0026lt;div className=\u0026#34;before\u0026#34; title=\u0026#34;stuff\u0026#34; /\u0026gt; // After \u0026lt;div className=\u0026#34;after\u0026#34; title=\u0026#34;stuff\u0026#34; /\u0026gt; The example above will only update the className prop of the div element, the same transformation will happen in the DOM only for the class-name attribute.\n// Counter.tsx const [value, setValue] = useState({ count: 0 }); \u0026lt;Counter value={value} /\u0026gt; // Update setValue({ count: 0 }); The example above will update the Counter component, because the object reference of the prop has changed. We could say, well okay, what if we passed a primitive instead? Surely React would compare it and not update the component if the value hasn\u0026rsquo;t changed.\n// App.tsx const [value, setValue] = useState(0); \u0026lt;Counter value={value} /\u0026gt; // Update setValue(0); The example above will update the Counter component, even though the value prop hasn\u0026rsquo;t changed, because the parent component (which holds the state) has re-rendered, and React will update the Counter component. But we can prevent this by using the memo hook.\nHave a look at this codesanbox that illustrate those behaviors\nNotice how the Object Ref with memo only re-renders once at the beginning, while the other components re-render every time you press the re-create value button.\nNote that this is only true for the first level of props (shallow comparison), if the prop is an object, React will compare the object reference, and if the object reference has changed, React will update the component.\nLists optimization When rendering lists, React will compare the list by reference, if the reference has changed, React will update the list. Using the memo hook on the list component will prevent the list items from updating if their props haven\u0026rsquo;t changed, which can be extremely useful when one has to update only a few items in a large list.\nSee the demo in codesandbox\nNotice how only the parent component re-renders when the list is updated, and the list items only re-render when their props change for the use memo implementation.\nRedux Toolkit optimization and common pitfalls Redux Tookit allows one to write Redux logic in a more concise and efficient way. It also comes with ImmerJs, which allows one to write reducers in a more readable and less error-prone way.\nWithout ImmerJs, one would write the following code to update a deeply nested object:\nfunction handleAction(state, action: Action\u0026lt;Substate\u0026gt;) { return { ...state, first: { ...state.first, second: { ...state.first.second, third: action.payload } } }; } ImmerJs allows one to write the above code like this:\nfunction handleAction(state, action: Action\u0026lt;Substate\u0026gt;) { state.first.second.third = action.payload; } Which is way more readable and less error-prone for behavior. Yet Immer has its own pitfalls and tradeoff:\nOverriding the entire slice Because Redux Toolkit uses ImmerJs under the hood, the state of our slices is wrapped in a proxy object (called draft). This proxy object will be used by Immer to derive a new state without mutating the current state while presenting a mutable facade to the developer.\nThis has a computational cost when creating the proxy object (of the magnitude of several seconds if it is big enough!), which is one of the many reasons why we want to avoid updating the entire slice when only a sub-state needs to be updated.\nConsider the following:\nfunction setHightlightedPhrase(state, action: Action\u0026lt;PhrasesState\u0026gt;) { state = action.payload; } As explained in the docs this will not work!. Here we only wanted to set the highlighted phrase, we ended up updating the entire slice and causing a re-render of all the phrases components. Plus the operation itself it will be slow if the action payload is large because of the proxy object wrapping.\nThis happened because ImmerJs will not compare deeply the new state with the old one. Assigning a new object to the state will merely change the reference of the state, and not the state itself. This is done for performance reasons, as it is costly to compare deeply nested objects.\nAs a general rule, ImmerJs WILL NEVER COMPARE DATA THAT IS ORIGINATING FROM OUTSIDE THE IMMER PROXY OBJECT WITH THE PROXY OBJECT ITSELF. Thus, only modify the leafs of the state, and not the state itself when applicable.\nIf one needs to reset, one needs to return a new object, like so:\nfunction updateArticleState(state, action: Action\u0026lt;ArticleState\u0026gt;) { return action.payload; } Which now no longer changes the reference of the state, but the state itself (recommended way by Redux Tookit docs).\nGenerally you should avoid updating the entire slice, and only update the sub-state that needs to be updated.\nCorrectly deriving data selectors createSelector is a powerful, composable and memoized selector creator. It accepts a list of input selectors and a transform function (also called output selector), and will only recompute the output selector if one of the input selectors has changed (this is checked with strict equality operator: ===).\nThis means that if an input selector is creating a new object, the output selector will always recompute:\nconst selectArticle = createSelector( [(state: RootState) =\u0026gt; state.article.map(article =\u0026gt; transformArticle(article))], // input selectors article =\u0026gt; article // output selector ); In the example above, a new array is created as a result of the map operator, and thus the output selector will always recompute. Note that this why the previous example of updating the entire slice is bad, because it will always recompute the output selector of anything that depends on the slice.\nTo fix this, move the transformation outside the input selector into the output selector:\nconst selectArticle = createSelector( [(state: RootState) =\u0026gt; state.article], article =\u0026gt; article.map(article =\u0026gt; transformArticle(article)) ); And now, the transformation will only happen if the article slice has changed. This is called memoization.\nShooting yourself in the foot with input parameters createSelector can accept input parameters, which can be useful for creating memoized selectors that depend on a parameter, say a props or a state. It is done like so:\nconst selectArticle = createSelector( [ (state: RootState) =\u0026gt; state.article, (_: RootState, articleId: string) =\u0026gt; articleId, ], (articles, articleId) =\u0026gt; articles.find(article =\u0026gt; article.id === articleId) ); And used like so:\nconst [articleId, setArticleId] = useState(\u0026#39;1\u0026#39;); const article = useSelector((state: RootState) =\u0026gt; selectArticle(state, articleId)); This is all well and good, but one needs to be careful with the input parameters. The above example would behave as expected BUT if the input parameter is a new object or a new reference, the output selector will recompute. This is because, as stated before, the input selectors verify equality of what they yield with the === operator.\nNow if you were to do this:\nconst articleIds = [1, 2, 3]; articleIds.map(articleId =\u0026gt; { const articleIdAsObject = { articleId }; const article = useSelector((state: RootState) =\u0026gt; selectArticle(state, articleIdAsObject)); // render something }); The output selector will always recompute, because articleIdAsObject will always be a new reference in the iteration thus the output selector will always recompute. Beware of input parameters coming from props, as they can be a new reference somewhere in the component tree.\nIf you really need to use an object as an input parameter, you can use the shallowEqual function from react-redux to compare the object references or, if, like me, you have a string primitive that react considered unchanged but redux didn\u0026rsquo;t, you can use createCachedSelector which will compare a custom cache key in place of the input selectors returned values.\nconst selectArticle = createCachedSelector( [ (state: RootState) =\u0026gt; state.article, (_: RootState, articleId: string) =\u0026gt; articleId, ], (articles, articleId) =\u0026gt; articles.find(article =\u0026gt; article.id === articleId) )( (_: RootState, articleId: string) =\u0026gt; articleId ); Calling immediately the selector DO NOT call a selector with the state like so:\nexport const selectForTranscriptionByLanguage = (language: string) =\u0026gt; {/*[...]*/}; export const combineLanguagesSelector = (state: ProvidersLanguagesVoicesState) =\u0026gt; { return { selectAvailableLanguagesPairForSuppliedLanguage: (language: string) =\u0026gt; createSelector( [selectForTranscriptionByLanguage(language)], (pairs) =\u0026gt; {/*[...]*/} )(state) /*[...]*/ } } This is bad because no memoization will happen, as a new selector will be created every time the combineLanguagesSelector is called.\nUse selector composition instead with input parameters:\nexport const selectForTranscriptionByLanguage = (language: string) =\u0026gt; {/*[...]*/}; export const selectAvailableLanguagesPairForSuppliedLanguage = createSelector( [selectForTranscriptionByLanguage, (_: RootState, language: string) =\u0026gt; language], (pairs, language) =\u0026gt; {/*[...]*/} ); References for further reading on the subject https://redux.js.org/usage/deriving-data-selectors \u0026lt;- VERY IMPORTANT! https://reselect.js.org/api/createselector/ Transforming efficiently large arrays or objects Avoid creating over and over objects:\nConsider the following problem: You have 3'000'000 values normalized between zero and one and you want to aggregate them in buckets of 100 values, taking the mean of each bucket. One could do it like so:\nconst values = Array.from({length: 3000000}, () =\u0026gt; Math.random()); const buckets = values.reduce((acc, value) =\u0026gt; { acc.currentBucket = [...acc.currentBucket, value]; if (acc.currentBucket.length === 100) { acc.buckets.push( acc.currentBucket.reduce((acc, value) =\u0026gt; acc + value, 0) / 100 ); acc.currentBucket = []; } return acc; }, { currentBucket: [], buckets: [], }); Which is of correct behavior but inefficient because of the array spread operator, which creates a new array every time a value is added to the bucket. This is costly because the array has to be copied entirely every time a value is added. Plus you may run into garbage collection pauses if the array is large.\nIt is better to use a simple loop:\nconst values = Array.from({ length: 3000000 }, () =\u0026gt; Math.random()); const buckets = []; let currentSum = 0; for (let i = 0; i \u0026lt; values.length; i++) { currentSum += values[i]; if (i % 100 === 0) { buckets.push(currentSum / 100); currentSum = 0; } } See the benchmark here. On my AMD Ryzen 9 5900X, the first solution took 680ms in median time to complete, while the second solution took 3.2ms in media time. That\u0026rsquo;s a 212x speedup!\nConclusion Avoid updating the entire slice of a Redux state, only update the sub-state that needs to be updated. Use createSelector to create memoized selectors, and be careful with input parameters. Avoid creating new objects in selectors, and use createCachedSelector if you need to compare objects with a custom equality function. When transforming large arrays or objects, avoid creating new objects or arrays, and use loops instead of array methods.\n","permalink":"https://blog.ultramaxu.me/posts/improving-react-apps-performances/","summary":"Disclaimer, this guide was done for my latest involvement with a React app, thus the examples are based on React 17 and thus this article will not address async rendering, suspense, etc. Also, the docs linked are from the legacy documentation for React 17.\nProblem Statement React, despite being one of the most popular front-end libraries, doesn\u0026rsquo;t have primitives, nor in the framework, nor in the underlying language to ward off performance issues.","title":"Improving React-Redux Apps Performances"},{"content":"But why? As I layed down the various requirements for my game engine, a requirement that seemed of vital importance was the ability to tests shaders and other graphical features. I live and breathe by TDD but I cannot do integration tests with a driver such as Vulkan or OpenGL (or maybe I can just not willing to do down this rabbit hole). An alternative is to do visual regression testing.\nThe plan is to pilot the Vulkan/OpenGL/WebGPU/whatever backend by wrapping it into a small executable and taking screenshots of the rendered scene. Then compare the screenshots with the expected ones. If they match, then the test passes. If they don\u0026rsquo;t, then the test fails. Easy.\nSo let me open a python project annnnd boring. I want the hard way! I have 700+ hours in Dark Souls 1. I love pain! So anyway, I\u0026rsquo;ve decided that X server would be my dev env and that I would write my own utility to take screenshots using Xlib.\nMy motivation for picking Rust for this task is that I want to evaluate how well it integrates with a C library. Also, Xlib is an old C lib, released in 1985, which makes it an idea candidate to evaluate Rust\u0026rsquo;s FFI capabilities with a lib that is not Rust friendly.\nI also want to see if Rust would be a strong candidate for my game engine.\n⚠️ DISCLAIMER ⚠️ If you didn\u0026rsquo;t gather already I am not that proficient with Rust. I\u0026rsquo;m just picking it as I go.\nXLib Now perhaps the most astute reader would ask despite that was stated before: \u0026ldquo;Why not use the XCB library? It\u0026rsquo;s a more modern way to interact with the X server and Rust has a crate for it.\u0026rdquo;\nWell as this post on stackoverflow states\nXCB is simpler to use, has a better response to a multithread environment but lacks documentation, while Xlib is a more dated/complex tool, better documented and fully implemented.\nWelp. I\u0026rsquo;m sold. I\u0026rsquo;m going to use Xlib.\nList of wanted features I want to be able to list windows I want to be able to take a screenshot of a window by its name Structuring the Hexagonal Architecture I love modularity. One day I\u0026rsquo;ll may have to integrate with other windowing systems. So I\u0026rsquo;m going to structure my app to be able to swap the Xlib implementation with another one - just in case.\nMy use case for taking the actual screenshot looks like this:\npub fn take_screenshot(\u0026amp;mut self, searched_window_name: String, output_path: String, ) -\u0026gt; anyhow::Result\u0026lt;ResultType\u0026gt; { let Some(target_window) = self.window_system_gateway.find_window(\u0026amp;searched_window_name)? else { anyhow::bail!(\u0026#34;Unable to find the window with title {:?}\u0026#34;, searched_window_name); }; let image_buffer = self.window_system_gateway.take_screen_shot(target_window)?; self.fs_gateway.save_image(image_buffer, \u0026amp;output_path)?; return Ok(ResultType::TakeScreenShotResult(())); } And As you can see, it is really not that complicated. Most of the heavy lifting is done by the Xlib adapter. We just check if we can find the window we want to screenshot, then we take the screenshot and save it to the output path. One lib that I allowed to creep its way into my core logic is the image crate, because I don\u0026rsquo;t want to manipulate raw bytes of a PNG. Maybe later.\nYou way wonder why I abstract so much behind traits. Well, it\u0026rsquo;s easy to test:\n#[test] fn it_should_work() { // Given let window_system_gateway = Box::new(FakeWindowSystemAdapter::new() .with_find_window_result(Box::new(|| Ok(Some(1)))) .with_take_screen_shot_result(Box::new(|| Ok(image::RgbImage::new(1, 1)))) ); let fs_gateway = Box::new(FakeFileSystemAdapter::new() .with_result(Box::new(|| Ok(()))) ); let mut usecase = TakeScreenShotUseCase::new( window_system_gateway, fs_gateway ); // When let result = when(\u0026amp;mut usecase); // Then assert!(result.is_ok()); } If the previous test is not clear, I\u0026rsquo;m using fake adapters that returns a predefined result. This way I can test when they fail:\npub struct FakeWindowSystemAdapter { find_window_result: Box\u0026lt;dyn Fn() -\u0026gt; anyhow::Result\u0026lt;Option\u0026lt;u64\u0026gt;\u0026gt;\u0026gt;, take_screen_shot_result: Box\u0026lt;dyn Fn() -\u0026gt; anyhow::Result\u0026lt;image::RgbImage\u0026gt;\u0026gt;, list_windows_result: Box\u0026lt;dyn Fn() -\u0026gt; anyhow::Result\u0026lt;Vec\u0026lt;String\u0026gt;\u0026gt;\u0026gt;, } impl FakeWindowSystemAdapter { pub fn new() -\u0026gt; Self { Self { find_window_result: Box::new(|| { Err(anyhow::anyhow!(\u0026#34;Unable to list windows.\u0026#34;)) }), take_screen_shot_result: Box::new(|| { Err(anyhow::anyhow!(\u0026#34;Unable to take screenshot.\u0026#34;)) }), list_windows_result: Box::new(|| { Err(anyhow::anyhow!(\u0026#34;Unable to list windows.\u0026#34;)) }), } } //... } #[test] fn it_should_report_finding_window_failures() { // Given let window_system_gateway = Box::new(FakeWindowSystemAdapter::new()); let fs_gateway = Box::new(FakeFileSystemAdapter::new()); let mut usecase = TakeScreenShotUseCase::new( window_system_gateway, fs_gateway ); // When let result = when(\u0026amp;mut usecase); // Then assert_error(result, \u0026#34;Unable to list windows.\u0026#34;); } And now that I have the structure in place, I can start writing the tests for the Xlib adapter.\nThe Xlib adapter Setting up a X server in a docker container While writing web apps, I\u0026rsquo;ve always tested database adapters by running said databases in docker containers. This provides a great deal of isolation and reproducibility. I\u0026rsquo;m going to do the same with Xlib. While reading the documentation, I\u0026rsquo;ve discovered that there exists an in memory X server called Xvfb (which stands for X Virtual Frame Buffer).\nSo the plan is: Spin a docker container with Xvfb:\nFROM alpine:3.20.1 ENV DISPLAY=:99 EXPOSE 6099 RUN apk add --no-cache xvfb openbox font-terminus feh xdotool RUN mkdir /images ENTRYPOINT Xvfb $DISPLAY -ac -listen tcp -screen 0 1024x1024x24 \u0026amp; openbox --debug And then I can run my test against this VERY ISOLATED environment. A couple of explanations about the dependencies:\nOpenbox is a window manager and I need it because well, without one, I wouldn\u0026rsquo;t be able to list windows ._. Font-terminus is a font that Openbox uses. Feh is an image viewer that I can use to display images that I\u0026rsquo;ll take screenshots of with the adapter. and about the ENTRYPOINT:\nDISPLAY specifies the display number of the X server. -ac disables access control restrictions so we can connect to the X server from any host (ofc unsecure but in ci it\u0026rsquo;s okay). -listen tcp allows Xvfb to listen on a TCP socket. We use it because the X server is running in a container and while we could mount the X11 socket of the host into the container, it\u0026rsquo;s way more convenient to use the TCP socket. That way you can develop this project on your machine even without X server running! -screen 0 1024x1024x24 specifies a screen of 1024x1024 pixels and depth of 24 bits (16.7 million colors). To test out our setup, we can start the docker container, connect to it and run a couple commands likexclock to start a window in xvfb and xwininfo (it lists windows) to see if we can interact with the X server.\ndocker build -t xvfb-alpine . # Xvfb listens on port 6000 for display 0, 6001 for display 1, etc. docker run --rm --name test-xvfb -it -p 6099:6099 xvfb-alpine export DISPLAY=\u0026#34;127.0.0.1:99\u0026#34; xclock xwininfo -root -tree Which should yield something like:\n0x200063 (has no name): () 166x189+429+417 +429+417 24 children: 0x40000a \u0026#34;xclock\u0026#34;: (\u0026#34;xclock\u0026#34; \u0026#34;XClock\u0026#34;) 164x164+1+20 +430+437 Nice. We can even take a screenshot with xwd of Xvfb:\nxwd -display :99 -silent -root -out image.xwd Alright. Now that we have a working X server, we can start writing tests for the Xlib adapter.\nTestcontainers in Rust It\u0026rsquo;s fairly standard usage really.\nenv::set_var(\u0026#34;DISPLAY\u0026#34;, \u0026#34;127.0.0.1:99.0\u0026#34;); // 1. let image_mount_dir = format!(\u0026#34;{}/tests/test_images\u0026#34;, env::current_dir().unwrap().display()); let container = GenericImage::new(\u0026#34;ultramaxu/ultramaxu-homelab-xvfb-alpine\u0026#34;, \u0026#34;0.0.0\u0026#34;) .with_wait_for(WaitFor::message_on_stdout(\u0026#34;Openbox-Debug: Moving to desktop 1\u0026#34;)) .with_mapped_port(6099, 6099.tcp()) // 2. .with_mount(Mount::bind_mount(image_mount_dir, \u0026#34;/images\u0026#34;)) // 3. .start() .expect(\u0026#34;Unable to start xvfb container\u0026#34;); This sets the DISPLAY environment variable to :99. We bind the port 6099 of the container to the host\u0026rsquo;s 6099 port. This is so we can connect to the X server from the host. We mount the images that we will display with feh during our test. Once the container is up and running, we can issue commands to it to run feh and display an image:\nfn start_feh_process(container: \u0026amp;Container\u0026lt;GenericImage\u0026gt;, title: \u0026amp;str, image_number: u32) { let command = format!(\u0026#34;feh --title {} /images/{}.png \u0026amp;\u0026#34;, title, image_number); // 1. let mut result = container.exec(ExecCommand::new( vec![\u0026#34;sh\u0026#34;, \u0026#34;-c\u0026#34;, command.as_str()])) .expect(\u0026#34;Unable to run the feh command\u0026#34;); // 2. for line in result.stdout().lines() { // 3. println!(\u0026#34;[STD OUT] {}\u0026#34;, line.unwrap_or(\u0026#34;[EMPTY LINE]\u0026#34;.to_string())); } for line in result.stderr().lines() { println!(\u0026#34;[STD ERR] {}\u0026#34;, line.unwrap_or(\u0026#34;[EMPTY LINE]\u0026#34;.to_string())); } thread::sleep(Duration::from_millis(100)); } The cool feature of feh is that we can specify a window title, perfect to test if the adapter can find a window by name. Now for the tricky part. You\u0026rsquo;ve noticed that I run feh in the background with the \u0026amp; operator. This is required so I may not have to await its completion (I\u0026rsquo;m using testcontainers sync api and I am not ready for async yet). Which means I have no means to know when the window is actually created. I\u0026rsquo;ve tried to use xdotool to wait for the window to appear\u0026hellip; to no avail! So I just sleep for a bit. Send to stdout the output of the command. This is useful to debug the container and avoid major headaches. By the way I love this image, because I\u0026rsquo;ve literally planted dynamite with this sleep. One day it\u0026rsquo;s going to blow up in my face. I\u0026rsquo;ll change it at some point.\nIN ANY CASE! Please have a look at the tests here before we move on to the actual implementation.\nThe Xlib adapter - the actual implementation Fortunately for us, I don\u0026rsquo;t have to call raw C directly, there is a lib that wraps Xlib for us: x11-dl. Well actually those are just bindings to the Xlib library. So we still have to deal with the C API. But in a convenient manner. This will be a nice introduction to how well Rust and C form a happy couple.\nOpening a connection to the X server, getting the root window Following the docs, the first thing to do is to open a connection to the X server, then get the root window handle (a pointer):\nunsafe { let display = x11::xlib::XOpenDisplay(std::ptr::null()); if display.is_null() { anyhow::bail!(\u0026#34;Unable to open X server display\u0026#34;) } let root_win = x11::xlib::XDefaultRootWindow(display); } XOpenDisplay: Honors the DISPLAY environment variable. Since we have set it in our test, it will connect to the Xvfb. Neat.\nTaking the screenshot As the documentation states:\nXGetImage. This function specifically supports rudimentary screen dumps.\nWhich sounds like exactly what I need! Though we need to feed it the window\u0026rsquo;s height and width. We can get those with XGetWindowAttributes:\nlet mut attributes: x11_dl::xlib::XWindowAttributes = std::mem::zeroed(); if x11::xlib::XGetWindowAttributes(self.display, window_id, \u0026amp;mut attributes) == 0 { anyhow::bail!(\u0026#34;Unable to get the window attributes of {:#x}\u0026#34;, window_id); } let width = attributes.width as u32; let height = attributes.height as u32; And now we can take the screenshot:\nlet image = x11::xlib::XGetImage( self.display, window_id, 0, 0, width as _, height as _, (self.xlib.XAllPlanes)(), x11_dl::xlib::ZPixmap as _, ); A bit of glossary first, the image we get from XGetImage is a pixmap. It is essentially a two-dimensional array of pixels. The pixel itself will be a ulong but can represent different things: like a single 8 bit value (monochrome), RGB(8,8,8) or RGBA(8,8,8,8) for transparency, etc.\nNow pixmaps can be worked on by the abstraction of planes. Each plane is a channel, like color of transparency. For example RGBA8888 is a pixel with 4 planes, each plane taking 8 bits. We have to specify a plane mask to tell xlib which planes we are interested in. The XAllPlanes function returns a mask that selects all planes, which is what we want (we want all the colors + transparency).\nNext we have to specify a format: ZPixmap or XYPixmap. Either way, we will get an integer, it\u0026rsquo;s just that XYPixmap can be used in combination of the plane mask to cull the planes we don\u0026rsquo;t want (like say we only want the R and B of the RGB). Again, we want all the colors, so we choose ZPixmap.\nMore about pixmaps in wikipedia.\nConverting the pixmap to an image To extract the individual color components (red, green, and blue) from a pixel in a ZPixmap format, we remeber that they are all stored contiguously in the pixel value. So we have to get the masks for each color so we can extract them:\nlet red_mask = (*image).red_mask; let green_mask = (*image).green_mask; let blue_mask = (*image).blue_mask; Horray: we have the masks. Now shift the pixel value to the right to get the actual color value and then pack the collected colors into a RgbImage (thanks to the image crate, I love you, whoever made it \u0026lt;3):\nlet mut imgbuf: image::RgbImage = image::ImageBuffer::new(width, height); for y in 0..height { for x in 0..width { // So pixel is a u_long let pixel = x11::xlib::XGetPixel(image, x as i32, y as i32); let r = ((pixel \u0026amp; red_mask) \u0026gt;\u0026gt; 16) as u8; let g = ((pixel \u0026amp; green_mask) \u0026gt;\u0026gt; 8) as u8; let b = (pixel \u0026amp; blue_mask) as u8; imgbuf.put_pixel(x, y, image::Rgb([r, g, b])); } } Ok(imgbuf) And that\u0026rsquo;s it. We have a screenshot of a window. Jesus wept that was a lot.\nBut haha. You think this is it? I mean, you can take the whole screen with this method by passing null as a window id. But what if you want to take a screenshot of a specific window? Well, you have to find it first.\nFinding a window Getting a name Arguably the harder part. And while you can get the window name with XGetWindowAttributes, don\u0026rsquo;t just don\u0026rsquo;t\u0026hellip; I tried. It worked on my computer™. But it didn\u0026rsquo;t work in the docker container, for some reason ¯\\_(ツ)_/¯.\nTo actually get a window name by its id, I had to look at the source code of xwininfo. So one has to try with XGetWMName function or, if that fails, with XFetchName.\nfn try_x_get_wm_name(\u0026amp;self, window: x11_dl::xlib::Window) -\u0026gt; Option\u0026lt;String\u0026gt; { unsafe { let mut prop: x11_dl::xlib::XTextProperty = std::mem::zeroed(); let ret = x11::xlib::XGetWMName(self.display, window, \u0026amp;mut prop); if ret == 0 { return None; } if prop.value.is_null() { return None; } let value = Some(CStr::from_ptr(prop.value as *const i8).to_str().unwrap().to_string()); (self.xlib.XFree)(prop.value as _); value } } fn try_x_fetch_name(\u0026amp;self, window: x11_dl::xlib::Window) -\u0026gt; Option\u0026lt;String\u0026gt; { unsafe { let mut data: *mut i8 = std::ptr::null_mut(); let ret = x11::xlib::XFetchName(self.display, window, \u0026amp;mut data); if ret == 0 { return None; } if data.is_null() { return None; } let value = Some(CStr::from_ptr(data as *const i8).to_str().unwrap().to_string()); (self.xlib.XFree)(data as _); value } } It feels funny to manage memory by hand in Rust. I guess it is the original sin™ lol. Both functions are called the same way: you initialize either a struct or a pointer to a char array, then you call the function, then you check if the return value is 0 (error) or if the pointer is null (error), then you convert the char array to a string, then you free the memory.\nIterating over a window\u0026rsquo;s children Before we can query the window name, we have to get the children of the root window. This is done with [XQueryTree](https://www.x.org/releases/current/doc/libX11/libX11/libX11.html#XQueryTree:\nunsafe { let mut root_return: x11_dl::xlib::Window = 0; let mut parent_return: x11_dl::xlib::Window = 0; let mut children: *mut x11_dl::xlib::Window = std::ptr::null_mut(); let mut nchildren: u32 = 0; if !x11::xlib::XQueryTree(self.display, window, \u0026amp;mut root_return, \u0026amp;mut parent_return, \u0026amp;mut children, \u0026amp;mut nchildren) == 0 { anyhow::bail!(\u0026#34;Unable to query the root window tree for window {:x}\u0026#34;, window); } if children.is_null() { // 1. return Ok(None); } let child_array = std::slice::from_raw_parts(children, nchildren as usize); // 2. for \u0026amp;child in child_array.iter() { let res = fun(child)?; if res.is_some() { // 3. return Ok(res); } } // Free the memory allocated for child windows if !children.is_null() { // 4. (self.xlib.XFree)(children as *mut _); } Ok(None) } If the pointer we get back is null then we return, it means this window is childless. We make a slice out of the pointer to be able to iterate over it, a read only view of this contiguous memory region, if you will. Now we can test if the child window is the one we are looking for. If it is, we return the window id. fun is a closure I pass to the function to either get the window name or to query the children of the child window. And we free the memory of course. I\u0026rsquo;ll forgo the rest of the implementation details, you can find it here. But essentially, now we only have to recursively look for the window we want, listing children each time and querying their names.\nConclusion Rust Definitely Cargo wins a place in my heart, it was so easy to just pull new dependencies and start using them (beats cmake and the like by a mile).\nThe borrow checker learning curve was a bit steep and the auto lifecycle of variables was a bit confusing at first. Like when I call make a variable like so:\nlet _ = run_xvfb_container(); And wonder why my container has died, only to realize that the variable was dropped immediately after the container was created which resulted in its destruction.\nRust and C marriage At one time I wanted to speed up converting pixmap values to RGB by using multithreading (overkill) but there I ran into the borrow checker wall. You see XGetImage returns a *mut XImage. One cannot pass this pointer to another thread because once cannot access a mutable reference or pointer from multiple threads - the borrow checker forbids it.\nI could copy the data array into sub-arrays and pass them to threads guaranteeing that they won\u0026rsquo;t overlap (which would please the borrow checker). And while this would work, a single threaded approach is fast enough for my use case. This highlights the two clashing philosophies between Rust and C apis such as Xlib.\nBecause Rust is about safety and C is more like \u0026ldquo;You know what you are doing.\u0026rdquo;\nSo Rust with FFI is a bit awkward. You\u0026rsquo;ll have to deal with C and write Rust compliant abstractions if you plan to use C libs.\nFor the context of game engines, that could be a limiting factor. Even though efforts are made to push Rust to the forefront with libs like WGPU apis like Vulkan or OpenGl are still the norm (and will be for many years).\nThere are libs like glium or vulkano to integrate such libs with rust. But as the docs of vulkano say:\nPlease note that by the current date none of the known projects in the ecosystem(including Vulkano) reached stable release versions and the final design goals, their APIs are changing from time to time in a breakable way too, and there could be bugs and unfinished features too.\nSo it\u0026rsquo;s a bit of a gamble at this moment.\nXlib Not much to say here. With examples and the doc it was reasonably easy to get what I wanted from it. I recommend https://cpp.hotexamples.com/ if you, like me, struggle to get a sense on how the lib should be used.\nClosing thoughts I think Rust is a strong candidate for building a game engine. I certainly see the appeal of building reusable blocks that can be swapped out while maintaining a contract between them, which is possible with Rust\u0026rsquo;s traits and the fact you can implement any trait for any type (really good).\nOverall the marriage wasn\u0026rsquo;t so bad and I got a working tool in the end.\nStay tuned for more adventures in game engine development!\n","permalink":"https://blog.ultramaxu.me/posts/taking_screenshots_with_xlib/","summary":"But why? As I layed down the various requirements for my game engine, a requirement that seemed of vital importance was the ability to tests shaders and other graphical features. I live and breathe by TDD but I cannot do integration tests with a driver such as Vulkan or OpenGL (or maybe I can just not willing to do down this rabbit hole). An alternative is to do visual regression testing.","title":"Taking screenshots with Xlib and Rust"}]