Posts by fs-support_HK

Rebuilding or reconfiguring the linux package will also trigger rsync again. So calling make linux-rebuild or make linux-reconfigure will also get your modified files updated in Builroot's local copy.

This is also true for other custom packages. Custom packages are useful in cases where you replace a given package by your own version of the package that is available in a separate directory and should not be downloaded by Buildroot. This is explained in the advanced section of the buildroot documentation, chapter 8.13.6.

https://buildroot.org/download…nual.html#_advanced_usage

Your F&S Support Team

The address where the device tree is loaded in RAM is in variable fdtaddr. To change this address, you have to change the fdtaddr variable and then you need to call run .fdt_nand again so that the fdt variable is also updated correctly.

Your F&S Support Team

Although there should be no dependency between the software and the toolchain version, the reality is different. There actually *are* some dependencies. For example newer toolchains do a more indepth analysis of the code to add more optimizations. However this has the "disadvantage", that those compilers may detect some irregularities in the code of older packages that were silently accepted in older toolchains. The new toolchain may issue lots of warnings or even errors then. On the other hand, a very old toolchain may not support all the language features of new packages (e.g. C++23) and may fail to build the package at all.

This is why the toolchain should always be of a roughly similar age as the packages it is used for. If the gap between the versions is very big, this condition may be difficult to fulfill.

We typically provide a new toolchain, if the userland packages of Buildroot need it, because there are new language features used by at least one package that the old toolchain was not capable to deal with. When configuring the toolchain, we typically provide headers so that at least the newest and the directly previous F&S release can be built. Otherwise also the headers and the support code increases and the toolchains get bigger and bigger.

There may also be dependencies between kernel and userspace packages. For example video deocding or OpenGL handling on NXP SoCs is split between a kernel driver and a userland library. And of course these two have to match properly. So if you use an older kernel with a new Buildroot rootfs, then these two may not cooperate correctly and the kernel may even crash. So be careful not to have a too big version gap between them.

We use crosstool-ng (https://crosstool-ng.github.io/) for our toolchains. After configuring the toolchain in a menuconfig-like selection system, you can start the build and then crosstool-ng will download all the source code packages and builds the toolchain. This is rather similar to Buildroots own concept.

Please note that there is also the possibility to let Buildroot build a toolchain or use a precompiled toolchain from other sources like codelinaro. This may be better related to your remaining build processes because it is not out-of-tree. Just play a little bit with the toolchain entry in the top menu of Buildroot's menuconfig.

Your F&S Support Team

Just to make sure: you really use a NULL modem cable with crossed signals? What serial speed do you use in the terminal program? Did you disable all flow control signals (RTS/CTS, XON/XOFF)?

Your F&S Support Team

The standard load address for the kernel is at 0x11000000 and the load address for the device tree is at 0x12000000. This has space for a kernel of up to 16 MB. If your kernel image is bigger, but you still load the device tree to 0x12000000, then the device tree will overwrite parts of the kernel and/or initramfs image. To avoid this, you have to change variable fdtaddr to 0x13000000 or similar.

Your F&S Support Team

The signals of UART_C are available on pins 23+25. The signals of UART_D are available on pins 18+20 (and 22+24 for RTS/CTS). In addition, one of the ports can also be routed to a level shifter and the signals are available with RS232 level on pins 36+38. It is decided by soldered 0-Ohm resistors, which one of the two ports is available on the level shifter.

By default, UART_C is routed to the level shifter. This means the signals of UART_C with RS232 level are available on pins 36+38. Of course in this case, the incoming signal UART_C_RXD is driven by the level shifter. If you are using the TTL level signal on pin 25, your incoming signal is working against the signal of the internal level shifter. The level shifter is holding the signal constantly at high, you are pulling it low in case of 0-bits. This results in a middle voltage of around 2V, depending on the driver strength of your adapter.

If you want to use both UART ports, you should use UART_C with RS232 levels on pins 36+38. Or contact our sales department to check if it is possible to provide a version without the connections to the level shifter.

Your F&S Support Team

We do not promote the ONOFF button because it is not tested on all boards and may not be supported in the default configuration of the software. As stated in the baseboard documentation, the function is only tested on the PicoCoreMX8MM and PicoCoreMX8MP.

The button is supposed to work like the power button of a PC. A short press is shutting down the system gracefully and a long press of 5 seconds is shutting down the system unconditionally. When it is off, pressing the button a short while, should repower the system.

The short press simply issues a key event, typically KEY_POWER, that can be handled by software. See node snvs-powerkey (label snvs_pwrkey) in imx6ul.dtsi. The long press simply shuts down the system and cannot be handled by software. This means a quick test if the button works at all would be a long press of 5 seconds. If the board shuts down, the button actually works.

Yocto (or more precisely, systemd) can handle the short press automatically but needs a modification in /etc/systemd/logind.conf.

The button itself is handled by a pin of the SoC located in the SNVS section. This means it also refers to the voltage level of this section. And yes, this is different on different F&S boards. On i.MX6UL, this is 3.3V, on i.MX8MM, this is 1.8V. As the button typically simply connects to ground (active low), this should not be a problem. However it can also be configured to be active high, and from the internal pull-down on the PicoCoreMX6UL100, I would assume that it is supposed to be configured in this way here. Then the button should connect to the SNVS voltage, but I'm not sure if this is available on the baseboard at all. Really strange.

I believe that this was never fully tested on PicoCoreMX6UL100 and I cannot guarantee that it fully works. Especially also the "ON" part of the button.

Your F&S Support Team

Thanks for the feedback.

Your F&S Support Team

Quote from bkeohane

I made the experience, that fbsplash and plymouth both do not start as early in the boot process as the penguins logos used to show up.

Yes.that's true. The penguins appear immediatley when the display driver is activated. fbsplash and plymouth cannot appear before the root filesystem is mounted, because they are taken from there. One solution may be a modification of the boot process. Currently, many of the drivers are built into the kernel image itself. By making all these drivers to kernel modules, the main boot process would speed up and the bigger part of the drivers would also be loaded later when the rootfs is available. This could reduce the time until the splash screen is visible in Linux, relative to the other drivers.

To speed up the boot process even further, it might be useful to introduce an initramfs as an intermediate root filesystem. This is how desktop Linuxes typically work. They start up with a rather small subset of drivers in the initramfs RAM disk, including fbsplash or plymouth. Because of the small size, this intermediate rootfs is mounted very quickly and you can see the splash screen very early in the boot process. Then the main rootfs is mounted from flash media, with all the remaining device drivers, which takes considerably longer. Then the rootfs is switched over from the intermediate rootfs to the main rootfs and the boot process of all the remaining drivers continues.

Having a smoother pass-over of the splash screen from U-Boot to Linux without the black gap inbetween is on our todo list for quite some time. But it is more difficult than it seems on the first glance. For example to be able to show the image that was visible in U-Boot, requires a memory region for the framebuffer that holds this image. This memory region has to be reserved in Linux, so that it is not used for regular tasks when booting up. How to provide such a reserved memory region at boot time but still be able to re-use it later on? Moreover, current Linux display drivers often do not use a framebuffer anymore at all, they use DRM. So how to pass on a framebuffer to a system that does not use a frambuffer? And if the image should be visible all the time, the display sub-system has to stay alive all the time. How to keep a subset of the clocks valid and enabled when Linux re-initalizes the whole clock tree? How to keep the display on when Linux re-initializes the display driver and backlight infrastructre in a completely different way?

We actually will have a look at this in the near future again, as we want to have display support in U-Boot for i.MX8 and i.MX9 CPUs, too. But I cannot promise that we will find an easy solution for all the mentioned problems.

Your F&S Support Team

When using splashimage, the image has to be a BMP image. If 8-bits do not work, try to use a 24-bit image. The display is using True Color anyway. As U-Boot is a rather minimalistic implementation, they may have spared the conversion from 8-bit with Color Lookup Table to 24 bit RGB in the code.

Replacing the penguins is not the correct way to handle a splash screen in Linux. The number of penguins tells the number of CPU cores. So if you simply replace the penguin with another graphic, you will also see this graphic twice or four times on a dual/quad CPU.

It is better to use a program like fbsplash to show a regular image or logo on the screen. This program is part of Busybox and is available on all our boards. It also supports showing a progress bar. We even have a document showing how to do this:

https://www.fs-net.de/filelice…fileid=11901&locale=de_DE

Your F&S Support Team

The board versions (or variants) are called V1, V2, V3, ... or with an added I in case of industrial temperature range, V1I, V2I, V3I, .... The board variant shows the equipped features of the board. The board itself can basically handle all variants, but we have also variants with less features that are less expensive. The variant decides whether there is WLAN or not, larger or smaller eMMC, more or less RAM, one or two LAN ports, etc.

The number found on the PCB is the board revision. This is independent from the version above. A new board revision is required if there was an improvement on the hardware, e.g. to fix a bug or to optimize something. Typically, new board revisions are only introduced rather early in the board development process and only if redesigns are required. After all issues are fixed, the board revision stays the same for the rest of the lifetime.

Revisions are numbered Rev1.xy. If the change is on the hardware side only, then y is increased. This is the case if a transistor needed to be replaced, a capacitor added, or the PCB routing was improved. Then for example Rev1.01 would change to Rev1.02. But if the new revision also has influence on the software side, then x is increased and y is reset to 0. For example if two GPIOs needed to be swapped, then also the software has to be changed to handle the new GPIO numbers. Then Rev1.02 would change to Rev1.10.

Your F&S Support Team

Yes, activating the display in U-Boot still works this way. However we have detected an issue in Linux when the display is activated in U-Boot. You need to apply the following patch to Linux or the display will not work in Linux afterwards.

In this solution, the screen still goes blank after U-Boot and is newly initialized in Linux. So the penguins are still there. But getting rid of the penguins is simple. Just add logo.nologo to the Linux command line, for example by adding it to the extra environment variable in U-Boot. The content of this variable is added to the command line.

This is also explained in the "Linux On F&S Boards" documentation in chapter 6.5.9.

Your F&S Support Team

Quote from WaSt

But the waveform shown in my last answer comes directly from the SoM Pin J1_54, which shouldn't be affected by +3V3 generated on the baseboard (PicoCoreBBDSI), right?

We have observed that the +3V3 voltage of the baseboard may be fed back via the serial level shifter to some GPIOs of the SoC. This in turn holds the SOM voltage VDD_3V3 on some middle value of around 1.5V, despite the fact that the SOM's PMIC is switched off. This 1.5V is high enough on the above EN pin of U22 to be detected as high, thus keeping the +3V3 of the baseboard running. So the idea of the additional pull-down is to pull the voltage below the trigger level of the EN pin, so that the +3V3 of the baseboard is safely switched off. Then also the VDD_3V3 of the SOM goes completely off.

Your F&S Support Team

A software partition is only required for boards that boot from eMMC, like the PicoCoreMX6UL. However, the PicoCOMA7 boots from NAND. So no partition is required, only MTD-partitions are necessary there, but they are completely handled in U-Boot. So command 'w' is not required (and also not available) there.

Your F&S Support Team

Basic release cycle

As our boards and modules are mainly based on System-on-Chips (SoCs) from NXP, we are also following the release strategy of NXP. That means most of our boards use the NXP-specific kernel. NXP always uses LTS kernels for their products, typically those that also have support for Android.

Why are we not using the mainline kernel? Well, when a hardware manufacturer like NXP develops a new SoC, they have to port and implement quite a lot of device drivers. This is a task of many months. If the kernel version would change during this time, this would be an additional difficulty. Therefore when they begin, NXP takes a then up-to-date mainline LTS kernel and then does all the porting on exactly this version until everything is finished. So after a year or so, NXP has a working kernel with all the support for the new SoC. This is the so-called manufacturer kernel, with still the same version as when NXP started. But in the meantime, the mainline kernel has released about six new versions. So at this point of time, the manufacturer kernel is at least six versions behind mainline, but it is the only kernel that has support for this new SoC. So when F&S designs new boards and modules with this new SoC, we depend on the manufacturer kernel, because we need its device driver support.

Then we port this code to our new boards based on this SoC, which again takes some time. Especially with the switch from 32-bit (i.MX6/7) to 64-bit CPUs (i.MX8), this was quite an effort, because the boot process got a lot more complex. There is not just U-Boot and Linux anymore. There is also the ARM Trusted Firmware (ATF), the Trusted Execution Environment (opTEE), the DDR4 DRAM has higher frequencies and requires board-specific timings now, there is a DRAM firmware, and some additional on-chip controllers like System Control Unit (SCU), security cores (SECO or Secure Enclave) and HDMI all need their own firmwares.

In the way how NXP has designed the boot process, the developer of the system software has to deal with all this low-level stuff, which would be you. And this is rather complex, because even within one board family, files like the DRAM timing may need different variants, depending on the RAM size and the RAM chip manufacturer. And you don't even have all the necessary information, because it is only relevant for us, the board manufacturer. For example you are simply not interested in what type of DDR RAM or eMMC Flash we use. But nonetheless you would have to deal with this. So NXP's way is not suited very well for cases, where the task is split in two parts, like us as board manufacturer and you as system integrator.

So we decided to do this slightly differently. We hide all the low-level stuff in a first-step bootloader called NBoot, that is already pre-installed on all our boards. This is only changed by us, you do not need to care about it. You only have to think about U-Boot and Linux. If you are familiar with our i.MX6 boards, you already know this, as it was the same there. So from the outside, the handling stays the same as on earlier boards, only internally, NBoot has completely changed. This makes it easier for you, and it allows us to continue providing architecture based releases, where only one version of NBoot/U-Boot is sufficient for all our boards based on the same SoC.

So after a while, we have our version of a kernel (and U-Boot), based on the NXP version. And of course these releases start with the newest boards. Then we do releases with this version for all our remaining platforms. And this is a lot of work: fsimx6 Buildroot and Yocto, fsimx6sx Buildroot and Yocto, fsimx6ul Buildroot and Yocto, fsimx8mm Buildroot and Yocto, fsimx8mn Yocto, fsimx8mp Yocto, fsimx8ulp Yocto and now also fsimx93 Yocto, just to name the most important ones.

In parallel, we start working on a new kernel and for new boards that are in the pipeline. This is typically the next NXP kenel, but if the previous development and all the releases took longer, we sometimes skip an NXP release. This explains the steps in the kernel number we have releases for. They are always the same as NXP's versions, but probably skipping an NXP version now and then.

Newest releases

Our current releases are based on kernel 5.15 on all our platforms. The neweset SoCs that we have are i.MX8ULP and i.MX93. Support for these chips was added after 5.15 in 6.1, so we made pre-releases based on the newer kernel 6.1. They are called pre-releases, because they are not yet merged with our regular software branch and they still use the original NXP way of booting. Our current work is to update our regular release version to the same version. However NXP has released 6.6 in the meantime, so we are skipping 6.1 as a regular release and are preparing all our next releases based on 6.6. Which is why we have a pre-relase for i.MX93 based on 6.6 already, but this is also still the NXP way of booting. The merge to NBoot is still in progress.

Unfortunately, the boot process for these new CPUs has changed again. That's why we had to improve our NBoot concept again to also support i.MX8ULP and i.MX93. This is finished now and we are preparing the regular releases for all our platforms based on kernel 6.6. We will start with i.MX93 and i.MX8ULP, because we only had pre-releases there and we want to have a regular release for them as soon as possible. Then i.MX8MP and all other architectures will follow. So for now, 5.15 is the newest version for i.MX8MP, but a 6.6 release is on its way and will be available by the end of the year.

Kernel patchlevel

NXP supports a release for a specific kernel version for a year or so, and during this time, they also update the patchlevel of the kernel version from time to time. But then after that, the patch level stays fix. This is why we decided, to update the kernel patchlevel on our own, independently of NXP. This is why you can find a newer patchlevel in our current releases.

Bringing patches upstream

We would actually like to bring back patches to the upstream projects. But there is an obstacle that we have not yet been able to overcome. NXPs releases are already some versions behind mainline, we are again some weeks or months behind NXP. But the mainline projects only accept patches relative to their newest versions. But we do not have these neweset versions running on our boards. So we actually never managed to prepare any patches that could be accepted by the upstream projects.

Fortunately, NXP is getting closer and closer to mainline. For example with U-Boot, they switch quickly to the same .04 version after mainline U-Boot is out. We also try to catch up. Our hope is to bring at least some of our changes to mainline U-Boot in the future, and maybe also to Linux some day.

Mainline kernel

Of course, after a SoC is in the market for some time, the SoC specific support gradually also migrates to the mainline kernel. Therefore, theoretically, we could also use the mainline kernel for our older boards. Up to now, we avoided this, to have exactly the same software version for all of our boards. But we are actually thinking about switching older architectures to mainline. This would allow users to update the kernel more independently from us. We will test this in the near future and maybe you'll also see mainline based releases for older boards, soon.

History file

Actually, the history file is obsolete now. In the past, our releases always updated all sub-packages, i.e. U-Boot, Linux and Yocto or Buildroot. So if we had a release for one architecture, and then a release for the next architecture, and there were no changes needed, we nonetheless had to update all the history files, resulting in new versions for all sub-packages.

Unfortunately, this concept made it rather difficult to have point releases as quickly as possible. A point release is a release, where only minimal changes are done, for example to fix a severe bug or add support for a new board revision. Here, changes are typically only in one sub-package, e.g. only in Linux. But we had to update *all* sub-packages, and it was not visible anymore, that for example U-Boot was the exact same version as before.

A year ago, starting with our 5.15 releases, we changed our release strategy considerably. We started providing our source code repositories on Github and combined the packages in a different way, pulling in the code from Github. From our internal point of view, all the sub-packages like U-Boot, Linux and Yocto/Buildroot are released on their own now, and a full release to the public, represented by a new releases-repository, only pulls in all the corresponding sub-packages. If we now do a point release that only differs in Linux, then we can pull in the exact same U-Boot as before, the exact same Yocto/Buildroot as before, and only need a newer Linux package. Unfortunately, this means we can not update the history file anymore, because the sub-packages do not know to which architecture release they currently belong or will belong in the future. In our example of the point release, U-Boot intentionally stays unchanged and therefore the history file, too. i.e. the history file cannot reflect the real history anymore.

We are still thinking of a good way of how to make the information of the history file available somewhere else. Part of it is in the Readme.txt of the release, part of it is the commit log of the git repositories. So we kept the history files for now, but they are obsolete and may disappear soon.

Your F&S Support Team

In fact yes. We plan to do a new Vybrid release. It currently competes with the new releases for i.MX93 and i.MX8ULP, but a new version is in the pipeline.

But it will be Buildroot only, we have no plans on porting Yocto to our Vybrid platform.

Your FS& Support Team

As you may know, the boot process of 64-bit-ARM CPUs is much more complicated. When using NXPs boot process (like in the pre-release), a lot of board-specific information like DRAM timings, has to be compiled into the source code. This would require a seperate bootloader for every board variant that we have, making our releases quite complicated. In addition, a lot of firmware images and other low-level stuff would have to be handled by the end customer.

So F&S has improved the boot process by introducing an intermediate bootloader layer called NBoot. This NBoot deals with all the low-level stuff and is already pre-installed on every shipped board. So like before on our i.MX6 boards, you just have to care about U-Boot an the Linux system.

All our boards based on i.MX8M CPU variants (MM, MN, MP) already use this NBoot based boot process. But on i.MX8ULP and i.MX93, the boot process is different. So at the moment we are porting NBoot to those new architectures. For i.MX93, this is already done and the next release will be based on NBoot. There are still some other things to do (Secure Boot, FreeRTOS, etc), but the release is not too far away. So right now we are porting NBoot to ULP, which works rather similar. So a rough guess would be to have a final i.MX8ULP release in October or November.

Your F&S Support Team

The Yocto version is basically ready. It is just the driver for the Silex BT/WLAN chip that still has problems. We are in contact with the manufacturer for quite some time, but there are still some issues. We are handling this with a rather high priority at the moment. As soon as this is sorted out, there will be a release.

Your F&S Support Team

The bad block is also reported in the log, "Bad eraseblock 484 at 0x000003c80000". But I do not think that this causes the problem. I believe the problem is the DMA timeout. In these cases, Linux fails to initialize the NAND, most probably because of a problem in the GPMI periphery (e.g. NAND DMA or NAND BCH engine). Access to NAND is done by setting up a sequence of DMA requests and then the sequence is executed, doing the real transfer from/to NAND flash. In these cases, the DMA sequence does not report success (or even completion). In other words, all accesses to NAND fail.

At the beginning, the first task of the NAND driver is to build a map of all bad blocks in RAM. This is done by accessing the frist page of each block in NAND. But as every access fails, the algorithm seems to mark every block as bad in the RAM table. This results in all those bad blocks that you see in this case. It does not have bad block markers on the NAND flash itself, it just recognizes all blocks as bad in the RAM table. When NAND access works again in a later boot process, again only the one real bad block is reported.

Unfortunately I have no idea why the initialization in Linux fails. Initialization in U-Boot did work, otherwise it could not have loaded and started Linux in the first place. Have you changed anything in the MTD/NAND code in your Linux version? Did you change any pad settings (driver strengths or similar) in the device tree regarding NAND flash? Or did you add a Realtime patch to Linux? Anything else?

Your F&S Support Team

NBoot ist not meant to be built by customers. Which is why there is no mannula how to do this. Nboot releases are done by F&S and all the NBoot version can always beo found in the F&S download sections. Each board is already shipped with a matching NBoot.

The reason is this: NBoot contains information about the board variants and parts that are mounted. If for example a DRAM chip goes end of life, then we can prepare a new variant with a replacement DRAM chip. So when you get the next shipment, there will be an NBoot installed on the board that can handle the new DRAM chip and you will notice no difference. But if you install your own NBoot version, then this is most probably not aware of the new DRAM chip yet. So the boot process may fail or the board will be instable due to wrong DRAM timings.

Of course the next regular Linux release will again hold all NBoot code to support the new DRAM chip. But this release may come out significantly later. NBoot releases are basically independent from Linux releases and may also happen between regular releases.

And here is another important rule: never downgrade NBoot to a version before the one that was shipped with the board. This may cause the same problem as above, tand the board may fail to boot are may behave strangely.

Your F&S Support Team