Hi,
This project looks great.
One question:
Niklas said:
>>
We also have an extensive concept for cooperative, stackless
multitasking, based on Dunkels Protothreads [19], which we expanded on
with Resumable Functions [20] which are used in all device drivers to
make communication non-blocking and allow multiple device drivers on
the same bus to safely access it.
This form of multithreading is very resource friendly since it does
not require context switches and only the main call stack (literally 2
bytes of RAM per Protothread).
However, this is quite an advanced topic and there are disadvantages
to this approach compared to a full real-time operating system (power
efficiency, hard real-time), such as mbed os or riot-os, however, they
do not necessarily work well on AVRs.
>>
Could you please expand on the disadvantages to power-efficiency or
hard real-time ? And how bad are those ?
Thanks,
Alex
Hi,
> - are the pins 'provided' by an extender as usable as normal GPIO pins (for instance to connect and LCD, or a cascaded extender)?
Coincidentally, the example described previously with the HD44780 driver over an I2C IO-expander is now officially an example.
https://github.com/roboterclubaachen/xpcc/blob/develop/examples/stm32f4_dis…
Cheers,
Niklas
Hi Wouter,
> On 27 Aug 2015, at 23:20, Wouter van Ooijen <wouter(a)voti.nl> wrote:
>
> Thanks for the long reply, which gives me some hints how to read your code. I'll probably come with some more questions later.
>
> You are of course welcome to post it on your mailing list!
>
> Do you know of other groups or individual who are using C++ in this let's call it 'advanced' way on small systems? I now know myself, Odin Holmes with his Kvasir, and you. (BTW I added you to http://www.voti.nl/blog/?page_id=144). I found your work by pure accident, so extrapolating from that there must be more. It would be nice to at least be mutually aware of all such efforts!
Yes, of course.
Christopher Kormanyos [1] wrote a book called “Real Time C++” [2] which is a very good generic introduction into the topic, but does not cover interface design as found in xpcc.
Christian Menard wrote an AVR C++ library call “yalla” [3], which is very impressive in its _consistent_ use of C++14. He also contributed to xpcc with a pull-request.
Andy has a very popular framwork called stm32cpp [4], which is however more “traditional” in its use of C++ and he builds on the CMSIS HAL layers provided by ST, which are written in C and this can limit the C++ interface.
He also has a very interesting TCP/IP stack written in pure C++ [5], which might be a good alternative for LWIP, and some really awesome hardware projects, like a FPGA sprite engine [6].
By far the most technologically impressive framework is written by Google Engineer C. Biffle, called ETL [7].
It only covers the STM32F407 and not all functionality of it, but his incredible in-depth knowledge of C++ and the amazing design of some of his APIs deeply impressed me.
I really need to append his implementation of typesafe register access [8] to our blog post about it.
There is also a OS kernel called Miosix [9], which aims to provide full C++ standard libraries, including posix compatible threading API, with a really cool scheduler. Only works on ARM, though.
A framework called McuCpp [10] written by Konstantin Chizhov most closely resembles xpcc.
He has very similar ideas of using static polymorphism for GPIOs and device drivers and even uses the same build system (SCons).
However, xpcc is a lot more complete and it is a lot easier to describe different peripheral hardware using our meta-template build system.
Those are all I remember from the top of my head, I hope I did not forget anything.
If you’re reading this on the list and have an awesome C++ µC library, just give it a shout-out.
Cheers,
Niklas
[1]: https://github.com/ckormanyos
[2]: http://www.springer.com/us/book/9783642346873
[3]: https://github.com/chrism333/yalla
[4]: https://github.com/andysworkshop/stm32plus
[5]: http://andybrown.me.uk/2013/07/14/stm32plusnet/
[6]: http://andybrown.me.uk/2014/06/01/ase/
[7]: http://cliffle.com/project/etl/
[8]: https://github.com/cbiffle/etl/tree/master/biffield
[9]: https://miosix.org
[10]: https://github.com/KonstantinChizhov/Mcucpp
Hi there,
Wouter wrote me an email regarding xpcc and below is my original long answer with a minor edit regarding connecting GPIOs in the configuration handler.
Cheers,
Niklas
> Hi Niklas
>
> I stumbled on your xpcc work, which seems very interesting. After a little brouwsing of the examples, sources and documentation I have a lot of questions about the basic architecture. Do you have any blogs or other documentation about that? Eaxmples of things I would like to know
>
> - how does the efficiency (speed, code size) compare to non-OO (plain C) code
> - do you use run-time objects for GPIO, LCD, extenders, etc, or do they 'vanish' at compiler time
> - is configurability limited to compile-time, or can you change for instance the I2C pins at run-time?
> - are the pins 'provided' by an extender as usable as normal GPIO pins (for instance to connect and LCD, or a cascaded extender)?
> - what kind of error messages does the user get when inappropriate (template) arguments are provided?
>
> An idea of what I am doing:
> - https://www.youtube.com/watch?v=k8sRQMx2qUw
> - www.voti.nl/blog
>
> best regards,
> Wouter van Ooijen
Hi Wouter,
this is going to be a long reply, with the idea that with your permission I would like to post this reply on our mailing list [1] for the benefit of our other developers.
[1]: https://mailman.rwth-aachen.de/mailman/listinfo/xpcc-dev, http://blog.gmane.org/gmane.comp.hardware.arm.cortex.xpcc.devel
> An idea of what I am doing:
> - https://www.youtube.com/watch?v=k8sRQMx2qUw
> - www.voti.nl/blog
I’ve seen your talk, and I agree 110% with your ideas.
The GPIO (and most other peripheral) implementation in xpcc is based on the same principles of static polymorphism as you described, however expands on these concepts quite extensively.
We have held a talk at FOSDEM’14 [2], however, with only ~20min we could only touch the surface, and we did not explain the solutions as well as you did (we were young and dumb ;-). The talk also describes our build system for generating and customising all peripheral classes out of our meta-template library.
I think it is very difficult to describe the advantages of this concept to developers who are not C++ experts or do not have a theoretical background (ie. software patterns) in software engineering.
Typically computer scientists or software engineers do not program microcontroller software.
[2]: https://archive.fosdem.org/2014/schedule/event/the_xpcc_microcontroller_fra…
> I stumbled on your xpcc work, which seems very interesting. After a little brouwsing of the examples, sources and documentation I have a lot of questions about the basic architecture. Do you have any blogs or other documentation about that?
We have a doxygen documentation that is relatively complete [3], however, it is not the right format to describe conceptual ideas.
Therefore we have a development blog [4], which goes very much into the details of the problem and presents our solutions in a more scientific way than just doxygen.
An interesting derivation on the concept of static polymorphism is how we compute baudrates at compile time.
This is actually an object-oriented approach, however, doesn’t even exist at runtime, neither as memory nor as code.
Most embedded C developers declare this to be black magic ;-)
There are several more topics that we want to describe [5] as such detailed blog posts, but of course, we are still students and we spend most of our time building robots, so the progress is slow on this xpcc “side-project”.\
[3]: http://xpcc.io/api/modules.html
[4]: http://blog.xpcc.io
[5]: https://github.com/roboterclubaachen/xpcc-blog/wiki
> Eaxmples of things I would like to know
>
> - how does the efficiency (speed, code size) compare to non-OO (plain C) code
This is a difficult question, since most concepts are not directly transferable to plain C code, or you wouldn’t use them like that.
There is also link-time or whole-program optimizations, which can produce some amazing results.
I recommend these slides [6] from for an in-depth exploration of C vs. C++ and what kind of “overhead” can be (not) expected. They are in German, I cannot find the English version.
xpcc started as a API for AVRs and for a long time we used AVRs to build our Eurobot robots [7]. So there was a real-world requirement to create a very efficient API for it.
Since we are one of the very few if not the only framework that supports both AVR and ARM with the same API, the efficiency also translates to the ARM implementations.
In practice this means:
- short functions (one-liners, such as Gpio set/reset) are forced to be inlined whenever possible,
- virtual functions are not used in the platform driver API, ie. full static polymorphism,
- complex computations are moved to compile time, like the aforementioned baudrate computation, and
- specialisation and customisation happens through code generation from our meta-template library, using device files as source [8].
This generates very efficient code in the sense, that the code requires none to very little RAM and if very fast due to inlining and compile time computations. Also note that there are no templates involved here, since we “manually” generate the classes that the compiler would generate out of templates.
For example, here are all available GPIO classes on the ATmega328 [9], or the STM32F407 [10].
This allows us to extensively specialise the peripheral drivers at zero cost. An example of this are the GPIO `connect` methods, that are generated from the information in the device drivers [11] and allow a typesafe and optimized setting of the pins alternate functions [12].
This is not possible with templates, however, especially useful when dealing with inconsistent hardware memory maps, as found on many AVRs, which do not have consistent register definitions [13].
We confirm the most timing critical implementations, such as GPIO, by counting cycles (at least on Cortex-M3s with built-in Trace Units) and/or looking at the generated assembly.
The fact that we can run the same code on an ATtiny that we run on an STM32 speaks for the low memory footprint.
[6]: http://aristeia.com/TalkNotes/C++_Embedded_Deutsch.pdf
[7]: https://www.youtube.com/watch?feature=player_detailpage&v=K7obV0avUoQ#t=259…
[8]: https://github.com/roboterclubaachen/xpcc/blob/develop/src/xpcc/architectur…
[9]: http://xpcc.io/api/group__atmega328p__gpio.html
[10]: http://xpcc.io/api/group__stm32f407vg__gpio.html
[11]: https://github.com/roboterclubaachen/xpcc/blob/develop/src/xpcc/architectur…
[12]: http://xpcc.io/api/structxpcc_1_1stm32_1_1_gpio_output_a2.html#a922dc0b8d94…
[13]: https://github.com/roboterclubaachen/xpcc/blob/develop/src/xpcc/architectur…
> - do you use run-time objects for GPIO, LCD, extenders, etc, or do they 'vanish' at compiler time
GPIOs as well as almost all peripheral drivers have no run-time representation, so their “objects" do indeed “vanish” at compile time.
However, communication and bus interfaces do contain static data, such as Uart [14] (which uses two atomic buffers for rx, tx) and the SpiMaster [15] and I2cMaster [16] (which manage their own resources and allow for safe multi-device access).
In particular, the I2C device drivers are quite complicated under the hood, since they allow a generic protocol customization at run-time using virtual function calls [17]. You need this for this custom EEPROM driver, which performs one I2C write for the address and then another for the data, without copying the data [18].
We also have an extensive concept for cooperative, stackless multitasking, based on Dunkels Protothreads [19], which we expanded on with Resumable Functions [20] which are used in all device drivers to make communication non-blocking and allow multiple device drivers on the same bus to safely access it.
This form of multithreading is very resource friendly since it does not require context switches and only the main call stack (literally 2 bytes of RAM per Protothread).
However, this is quite an advanced topic and there are disadvantages to this approach compared to a full real-time operating system (power efficiency, hard real-time), such as mbed os or riot-os, however, they do not necessarily work well on AVRs.
So there are are many types of “objects" and different definitions of “overhead”.
Of course, using virtual functions in the I2cTransaction is an overhead, however, the complexity of the protocol demands it.
Of course, the resource management requires a few more bytes (literally), however, the nature of multitasking demands it.
Of course, the protothreads are polling based, however, there is no real alternative for light-weight multitasking the AVR.
[14]: http://xpcc.io/api/classxpcc_1_1_uart.html
[15]: http://xpcc.io/api/classxpcc_1_1_spi_master.html
[16]: http://xpcc.io/api/classxpcc_1_1_i2c_master.html
[17]: http://xpcc.io/api/classxpcc_1_1_i2c_transaction.html
[18]: https://github.com/roboterclubaachen/xpcc/blob/develop/src/xpcc/driver/stor…
[19]: http://xpcc.io/api/group__protothread.html#details
[20]: http://xpcc.io/api/group__resumable.html#details
> - is configurability limited to compile-time, or can you change for instance the I2C pins at run-time?
In general no, but if the hardware allows for multiple I2C pins at the same I2cMaster then yes.
So for example, the SoftwareI2cMaster class [21] bitbangs the protocol on two pins as template parameters, which cannot be changed at runtime.
Of course you could use a second SoftwareI2cMaster class for the other two pins, but you cannot move your device driver over to this bus.
typedef xpcc::SoftwareI2cMaster<GpioB1, GpioB2> I2cMaster1;
typedef xpcc::SoftwareI2cMaster<GpioB3, GpioB4> I2cMaster2;
xpcc::Pca9535<I2cMaster1> expander; // device always uses B1, B2
This is a little different for Hardware I2C, since the pins alternate function is configured at runtime in hardware:
typedef xpcc::stm32::I2cMaster1 I2cMaster1;
xpcc::Pca9535<I2cMaster1> expander; // device always uses I2cMaster1
GpioB1::connect(I2cMaster1::Sda); // remember the connect method from earlier?
GpioB2::connect(I2cMaster1::Scl);
I2cMaster1::initialize<...>();
// expander uses pins B1, B2
I2cMaster1::reset(); // stop all transactions
GpioB3::connect(I2cMaster1::Sda);
GpioB4::connect(I2cMaster1::Scl);
I2cMaster1::initialize<...>();
// expander uses pins B3, B4
The same is possible for SPI and UART, if the pins have the right multiplexes of course!
However, there is actually a much more interesting problematic going on with this interface.
What do you do if you have multiple devices on a bus, and they have different settings?
In xpcc you can reconfigure the bus on the fly using configuration callbacks [22]:
void fastSpeed() {
SpiMaster::initialize<systemClock, MHz20>();
SpiMaster::setBitOrder(SpiMaster::BitOrder::LsbFirst);
}
void slowSpeed() {
SpiMaster::initialize<systemClock, MHz1>();
SpiMaster::setBitOrder(SpiMaster::BitOrder::MsbFirst);
}
xpcc::FastSpiDevice<SpiMaster, GpioB2> fastDevice; // GpioB2 = Chip Select
xpcc::SlowSpiDevice<SpiMaster, GpioB3> slowDevice;
// attach configuration callbacks:
fastDevice.attachConfigurationHandler(fastSpeed);
slowDevice.attachConfigurationHandler(slowSpeed);
Now, whenever any device accesses the bus, the SpiMaster calls its configuration handler to reconfigure itself.
The same is possible for the I2cMaster.
Notice how this moves the platform-specific code, namely the speed and bit order, into the user code, while still remaining generic for the device driver.
EDIT: You can of course also re-connect your GPIOs in the configuration methods!
void slowSpeed() {
GpioB3::connect(I2cMaster1::Sda); // yes, this works!
GpioB4::connect(I2cMaster1::Scl);
SpiMaster::initialize<systemClock, MHz1>();
SpiMaster::setBitOrder(SpiMaster::BitOrder::MsbFirst);
}
[21]: https://github.com/roboterclubaachen/xpcc/blob/develop/src/xpcc/architectur…
[22]: http://xpcc.io/api/classxpcc_1_1_i2c_device.html
> - are the pins 'provided' by an extender as usable as normal GPIO pins (for instance to connect and LCD, or a cascaded extender)?
There is no concept for that in our code generation, however, we have had to solve this issue before, when we wanted to drive a HD44780 display [23] using a I2C IO-expander.
We suddenly had the problem, that we couldn’t pass the I2C expander to the LCD driver, since we never assumed that GPIOs could be dynamic ;-)
So the solution was to create a class with the same static functions as expected by the driver and wrap the I2C expander like this:
extern xpcc::Pca9535<I2cMaster1> expander; // either extern or static
class GpioExt4 {
…
static void set() {
expander.set(Pin::P4);
}
}
Let me be clear: This is not efficient, due to the I2C bus access and its blocking! But it worked.
I haven’t tried passing these GPIO classes to our SoftwareI2cMaster so that we can cascade another I2C expander as you suggested in your talk, but it might work ;-)
[23]: http://xpcc.io/api/classxpcc_1_1_hd44780.html
> - what kind of error messages does the user get when inappropriate (template) arguments are provided?
C++ Template errors are really terrible and cryptic. I don’t know why compiler developers hate us so much.
We mainly use `static_assert` to confirm some basic functionality.
An example would be the width of the bus in our `GpioPort` classes [25] or directly asserting the correct bus width in the HD44780 driver [26]. And of course there is the afore-mentioned baudrate computation.
Unfortunately there is no way of confirming that a Template parameter confirms to a specific interface, however, this functionality called “ConceptC++” was actually a C++0x proposal [27], but it did not make it into C++11 nor C++14. Maybe C++17?
[24]: https://www.youtube.com/watch?v=mGS2tKQhdhY
[25]: https://github.com/roboterclubaachen/xpcc/blob/develop/src/xpcc/architectur…
[26]: https://github.com/roboterclubaachen/xpcc/blob/develop/src/xpcc/driver/disp…
[27]: http://www.generic-programming.org/languages/conceptcpp/tutorial/
Kind regards,
Niklas
Hi there,
I’ve rewritten the STM32 linkerscripts to make use of the Core-Coupled Memory whenever possible.
To place static variables into the CCM use `ATTRIBUTE_FASTDATA`.
Similarly you can move functions into CCM by using `ATTRIBUTE_FASTCODE`, but be aware that this is only supported on the STM32F3.
On all other devices both attributes revert back to the normal SRAM, which might still be faster than flash or fetch with less jitter.
For a detailed description see this merge commit [1].
Two new discovery boards are now supported:
- STM32VL discovery (STM32F100), and
- STM32F429 discovery.
See the amazing blinking LEDs in the examples folder.
Fortunately the peripheral IPs were very similar or identical to the one we already implemented, so they should work without much trouble.
There ist also theoretical support for the STM32F7 discovery, however I cannot get OpenOCD to flash my STM32F7 discovery, so it still has to be proven.
The STM32F401 Nucleo has also seen some blinking LEDs and functioning UART, but is not yet available in the examples.
Cheers,
Niklas
[1]: https://github.com/roboterclubaachen/xpcc/commit/567fe672f55f4e177d10f450c8…
Hi there,
xpcc can now automatically generate the correct interrupt vector tables for all STM32 devices.
The table is generated from the CMSIS headers and stored in the Device Files.
Example STM32F407:
https://github.com/roboterclubaachen/xpcc/blob/develop/src/xpcc/architectur…
This should dramatically improve the coverage and correctness of all interrupts on the STM32 devices.
Some small inconsistencies in the device files, such as wrong SRAM size or duplicate values, have been fixed as well.
Furthermore all STM32 include files have been updated to the latest from the ST Cube libraries.
Cheers,
Niklas
Hi,
xpcc has had a very exciting year with the introduction of Resumable functions and the following improvements of almost all drivers to be non-blocking (see the last change logs).
We (the RCA [1]) have used xpcc in our two robots to achive the Vice Championship in the International Eurobot Competition in Switzerland, so that was very exciting and a lot of fun.
The major refactoring that we started about 2 years ago to generate the architecture library out of templates using our device files [2] has proven itself to the right tool to deal with the many small differences between peripheral drivers of a microcontroller family.
However, the integration of this highly custom solution into our build system SCons with a lot of custom code is getting to the point where we are fighting SCons more than we should and this is always a sign that something is wrong.
In fact,we wanted to uncouple the generation of the custom framework out of our templates from the actual build process for a long time now, since that would also enable us to use a build system other than SCons to compile and link the application code.
@dergraaf has been working on a new library builder concept which will deal only with the framework generation and customization and module composition, but not compilation.
Under with concept, we can group functionality as modules and declare dependencies between them.
So the `xpcc/architecture` folder would be one module, `xpcc/ui` might be another module.
Your project will have a configuration file declaring which modules are needed and how they should be configured.
We will continue this development of xpcc under a new name, since XPCC is originally the name for the IPC protocol that we use in our robots [3] and it has always been awkward that the framework was named after it.
The new framework will be licensed under GPL with a linking exception and will be called `modm` (pronounced modem).
modm stands for `modular object-oriented development for microcontrollers` [4].
xpcc.io and its repositories will remain in place until this transition is completed, hopefully until the end of this year.
Then xpcc will stand for the protocol and modm for the framework.
Cheers,
Niklas
[1]: http://www.roboterclub.rwth-aachen.de/cms/
[2]: https://github.com/roboterclubaachen/xpcc/blob/develop/src/xpcc/architectur…
[3]: http://xpcc.io/api/group__xpcc__comm.html
[4]: https://github.com/roboterclubaachen/modm
Hi folks,
I just want to say hello and tell you what an amazing work you do :). With
xpcc you achieved exactly what I tried to to build for the last ~6 years, but
I never had the time nor the resources to actually get something useful.
Why is this project not more popular? I only stumbled across by chance. I
think it deserves allot more attention. Maybe some advertisement in the big µC
forums would help?
You will probably hear more from me in next time as I will use xpcc for
various projects. I will also gladly contribute my work. For starters I wrote
a driver for the PCA9685 PWM controller and I'm just about to create a pull-
request.
Cheers
Christian
