tag:blogger.com,1999:blog-260284912024-03-13T22:20:21.381-07:00Random Encounters&ThoughtsThomas Maier-Komorhttp://www.blogger.com/profile/10885556724411617546noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-26028491.post-61305367163913413732020-02-11T10:58:00.001-08:002020-02-11T10:58:03.380-08:00Data Serialization - Getting started with Wire Format Compiler<h2>
Getting started with Wire Format Compiler</h2>
<div>
Data serialization is the process of getting an in memory data structure into a stream of bytes that can be sent via a network connection or written to disk and back. For PCs, servers, and smart phones there are many options to do this. There is no wrong or right way to do it, but different concepts offer different advantages and disadvantages.</div>
<div>
<br /></div>
<div>
For embedded systems <a href="http://github.com/maierkomor/wfc">Wire Format Compiler</a> is a tool that generates C++ source code that performs this task. The generated code is light-weight enough to run on small embedded systems. Its language has been inspired by Google's protocol buffers, but it offers numerous enhancements for use on embedded systems, and provides about 3-4 times better performance when it comes to processing time and ROM footprint. Additionally, with WFC you are able to even target very small embedded systems such as 16 bit devices or even 8 bit devices. For this many options exists that allow to tune the generated code by taking advantage of specific aspects of small devices, such as lower address space, register width or implementation concepts.</div>
<div>
<br /></div>
<h3>
First simple Example</h3>
<div>
As a first example let's take a look at the configuration data of a potential embedded device. The data description in WFC source format looks like this:</div>
<div>
<br /></div>
<div>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;"><span style="color: red;">message</span> Config<br />{<br /> <span style="color: blue;">string</span> hostname = 1;<br /> <span style="color: blue;">unsigned</span> baudrate = 2;<br /> <span style="color: blue;">string</span> wifi_ssid = 3;<br /> <span style="color: blue;">string</span> wifi_pass = 4;<br /> <span style="color: blue;">bool</span> station = 5;<br /> <span style="color: blue;">fixed32</span> ipv4 = 6;<br /> <span style="color: blue;">fixed8</span> netmask = 7;<br /> <span style="color: blue;">fixed32</span> gateway = 8;<br />}</span></blockquote>
</div>
<div>
This description specifies a data structure with several members. First the data type is specified, second is the name of that structure member, and last the unique identifier. The unique identifier must never be changed after bringing data into real world usage. The data type can be modified for later software updates with some restrictions, that I may talk about in a later post. Backward and even forward compatibility is very strong for WFC generated code, if you follow some rules when extending messages for a later release. Forward compatibility is achieved by being able to skip over unknown data structures, but still extracting what is well known.</div>
<div>
<br /></div>
<div>
The data types used in this example are:</div>
<div>
<ul>
<li><span style="font-family: Courier New, Courier, monospace;">string</span>: this will be implemented as a <span style="font-family: Courier New, Courier, monospace;">std::string</span> per default, but this can be changed to a target specific more appropriate type. How to do this, I will write about in a different post.</li>
<li><span style="font-family: Courier New, Courier, monospace;">unsigned</span>: This data type will be generated as a platform specific unsigned integer data type. Take care to make sure that the default unsigned type has the appropriate range of values. This data type is serialized as a variable width integer that will require 10 byte at most to be stored plus usually 1 or 2 bytes for the message member tag.</li>
<li><span style="font-family: Courier New, Courier, monospace;">bool</span>: This is just the regular boolean type. Data serialization relies also on the concept of variable length integer, but boolean data will of course only take 1 byte for its data. Due to relying on the variable integer concept a boolean type can later be changed to an enumerated data type or some integer type that is not serialzed in signed format.</li>
<li><span style="font-family: Courier New, Courier, monospace;">fixed8</span> and <span style="font-family: Courier New, Courier, monospace;">fixed32</span>: These are fixed width integers that will be represented as <span style="font-family: Courier New, Courier, monospace;">uint8_t</span> and <span style="font-family: Courier New, Courier, monospace;">uint32_t</span> accordingly. <span style="font-family: Courier New, Courier, monospace;">Fixed8</span> will require 1 byte, <span style="font-family: Courier New, Courier, monospace;">fixed32</span> needs 4 bytes in their serial representation plus the tag. </li>
</ul>
</div>
<div>
Of course there are many more types. I will take a look at what other types are available in a later post.</div>
<div>
<br /></div>
<div>
Without any additional options WFC will generate code for serializing an deserializing the data structure to and from memory. Additionally, helper functions are generated to calculated the serialized size for a specific data instance, and to serialize the structure directly to a C++ std::string. Furthermore, options exist to also create an ASCII or JSON output for human interaction or interfacing with other tools.</div>
<div>
<br /></div>
<h3>
Integrating the generated code into an application</h3>
<div>
To integrate the generated code into your application, you need to include the generated header file into your code and add the generated C++ source file into your build process. To make use of the generated code, simply create an instance of the classes defined in the generated head file. In this case the class is called Config, just as the message is called.</div>
<div>
<br /></div>
<div>
So let's take a look how initializing, serializing, and deserializing work in practice. This examples is also included in the WFC source distribution and contains multiple variants how to use the generated code. Here we just take a look at the variant with C++ streams:</div>
<div>
<br /></div>
<div>
To initialize the structure, just set every member of it:</div>
<div>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;"><span style="color: blue;">void</span> initDefaults(<span style="color: blue;">Config</span> &cfg)<br />{<br /> cout << "initializing defaults\n";<br /> cfg.<span style="background-color: #93c47d;">set_hostname</span>("host001");<br /> cfg.<span style="background-color: #93c47d;">set_baudrate</span>(115200);<br /> cfg.<span style="background-color: #93c47d;">set_wifi_ssid</span>("mywifi");<br /> cfg.<span style="background-color: #93c47d;">set_wifi_pass</span>("1234567890");<br />}</span></blockquote>
</div>
<div>
Next let's serialize the data to a file:</div>
<div>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;"><span style="color: blue;">void</span> saveConfig(<span style="color: blue;">const Config</span> &cfg)<br />{<br /> <span style="color: blue;">string</span> str;<br /> <span style="background-color: #93c47d;">cfg.toString(str);</span> <span style="color: #3d85c6;">// serialize the data</span><br /> ofstream <span style="color: magenta;">out</span>;<br /> <span style="color: magenta;">out</span>.open(CfgName); <span style="color: #45818e;">// open file</span><br /> <span style="color: magenta;">out</span> << str; <span style="color: #45818e;">// write to file</span><br /> <span style="color: red;">if</span> (<span style="color: magenta;">out</span>.good())<br /> <span style="color: magenta;">cout</span> << "writing config successful\n";<br /> <span style="color: red;">else</span><br /> <span style="color: magenta;">cerr</span> << "error writing config\n";<br />}</span></blockquote>
</div>
<div>
Now that was also quite easy. We just serialize the data to a string and write the string to a file. That's it. So now the last important step comes. Deserializing the data:</div>
<div>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;"><span style="color: blue;">int</span> readConfig(<span style="color: blue;">Config</span> &cfg, <span style="color: blue;">const char </span>*CfgName)<br />{<br /> <span style="color: blue;">ifstream</span> in;<br /> in.open(CfgName,ios::in); <span style="color: #45818e;">/</span></span><span style="font-family: "Courier New", Courier, monospace;"><span style="color: #45818e;">/ open file</span></span><span style="font-family: Courier New, Courier, monospace;"><br /> in.seekg(0,ios::end); <span style="color: #45818e;">// seek to end...</span><br /> ssize_t n = in.tellg(); <span style="color: #45818e;">// to get the size.</span><br /> in.seekg(0,ios::beg); <span style="color: #45818e;">// back to begin</span><br /> <span style="color: red;">if</span> (n <= 0)<br /> return 1;<br /> <span style="color: blue;">char</span> *buf = new char[n]; <span style="color: #45818e;">// allocate buffer</span><br /> in.read(buf,n); <span style="color: #45818e;">// read data</span><br /> <br /> <span style="background-color: #93c47d;">cfg.fromMemory(buf,n);</span><span style="background-color: white;"> <span style="color: #45818e;">// parse the data</span></span><br /> <span style="color: red;">delete</span>[] buf; <span style="color: #45818e;">// clean-up</span><br /> // That's it! Ready to go.<br /> <br /> <span style="color: magenta;">cout</span> << "read config from " << CfgName << endl;<br /> <span style="color: red;">return</span> 0;<br />}</span></blockquote>
</div>
<div>
Reading back is more complex, because we need to know how much memory to allocate and the C++ streams interface has no easy way to determine the size of a file. WFC even has a concept that enables it to find the end of the stream by itself. But this is an extension that I do not want to dive into here, because it needs some additional considerations. Restoring the serialized data is also just one line that asks the relevant class object to parse the data from memory. What is done behind the scene can be found in the code generated by WFC. But you don't have to understand anything about it, to use the generated code.</div>
<div>
<br /></div>
<div>
That's it for this post. The next posts will take a look at more data types, how to do some target specific optimizations, and how to use custom string types to reduce the footprint even further.</div>
<div>
<br /></div>
<div>
Get <a href="http://github.com/maierkomor/wfc">Wire Format Compiler</a>. Happy hacking!</div>
Thomas Maier-Komorhttp://www.blogger.com/profile/10885556724411617546noreply@blogger.com0tag:blogger.com,1999:blog-26028491.post-48810700706235166712020-02-11T06:13:00.000-08:002020-02-11T06:13:17.717-08:00Automatic Versioning with Make/CMake and Mercurial<h2>
Keeping Version Strings Up-To-Date</h2>
<div>
A software package and its binary executables should have a version number or a release name. This is more or less common practice, due to the benefits one has, when it is possible to identify the version of a specific software.</div>
<div>
<br /></div>
<div>
But what is the best way to bring the version tag into the software and keeping it up-to-date? Traditionally, version information has been hard coded into the source code. This is the simplest and most straight forward approach, but this comes with a big question: When and how do you update the version to e.g. version 2.0?</div>
<div>
<br /></div>
<div>
It makes sense to do it in an exclusive commit to the revision control system of the project. Additionally, it should be done after performing all those release and regression tests. But if the version string is hard coded, you will be unable to figure out from which commit of your repository a specific binary was built, unless you update the version string on every commit. But if you cannot ensure that only the latest and greatest binary will be used, it can be quite important to identify to which commit a specific binary relates during development.<br />
<br />
Furthermore, if you change the hard-coded version string after release testing, how do you know you didn't break the code while changing the version string? So another release test of the version with the correct string might become necessary. And, as we all know, manual tasks are prone to error. So sooner or later, manually updating the version string will be forgotten, the update will be incorrect or introduce some bug. Maybe the code will not compile or even worse the version string will be incorrect or whatever...<br />
<br />
So the simple hard-coded version string concept may work well in many cases, but might not make everybody happy. </div>
<div>
<h2>
Automatic Version Generation</h2>
</div>
<div>
Therefore, it makes sense to think about a better solution. A better approach is to generate the version string during build time. This concept also can provide more information about the build itself. E.g. hostname of build server, version of tools used for building, time of build and so on come to mind.</div>
<div>
<br />
For this, support from the build system and the version control system is necessary. The build system must trigger an update of the relevant information and link it to the binary.</div>
<div>
<br />
But how can this information be gathered, when it should be available at build time, but not hard-coded into the sources? One way to do it, is to use the infrastructure of Mercurial as a revision control system and its tagging mechanism. The advantage of this concept is that it also works for archives that have been generated by mercurial, but have no reference to the repository. With other revision control systems you might have to come up with another approach, but probably will find a similar solution.</div>
<div>
<br /></div>
<div>
Using <a href="https://www.gnu.org/software/make/" target="_blank">GNU Make</a> or <a href="https://cmake.org/" target="_blank">CMake </a>as build systems, Mercurial's infrastructure can easily be employed to provide the necessary information. Mercurial's tags provide the ability to associate a given commit with a version string. Like this a specific revision can be given a name or version <b>after </b>its commit has been submitted and tested. So you can do the commit, test it, and once you are sure all release prerequisites are fulfilled and it is ready for publication, you tag the revision with a version name without changing any code manually. This reduces the risk of breaking anything dramatically.<br />
<br />
Furthermore, Mecurial also lets you query the distance to the latest tag. So if the latest tag is always the latest version number, the delta can be used as a patch level to the named version. For a mercurial repository the latest tag and distance as patch level can be queried with '<span style="font-family: "courier new" , "courier" , monospace;">hg log -r . "{latesttag}.{latesttagdistance}"</span>'.<br />
<br />
Now, if the repository gets exported to a zip or tgz archive, the repository cannot be queried anymore. The good thing is that Mercurial creates a file called <span style="font-family: "courier new" , "courier" , monospace;">.hg_archival.txt </span>that contains just this information. To extract the version information from this file some shell scripting with <span style="font-family: Courier New, Courier, monospace;">awk</span> or <span style="font-family: Courier New, Courier, monospace;">grep</span> and <span style="font-family: Courier New, Courier, monospace;">sed</span> is necessary. All this is demonstrated here in a sample repository with a small shell script, which should work on all UNIX based systems like Linux, BSD, Solaris or MacOSX.<br />
<br />
The version string itself must be written either to a header file or a source file for make and cmake being able to recognize a change in the version string and trigger the appropriate build steps, unless you want to do a full build every time. So putting the version string in a command line argument for a <span style="font-family: "courier new" , "courier" , monospace;">#define</span> will not yield the intended result, as a source code delta might be overlooked by that approach by GNU Make and CMake. Therefore, just make sure to generate a source file that is picked up during the build process with all relevant version information that you would like to have integrated into the binary.<br />
<h2>
Example</h2>
Let's take a look at a small example, how to use the template repository. The demo <a href="http://www.maier-komor.de/software/mkversion.hg">repository</a> contains a file called <span style="font-family: Courier New, Courier, monospace;">hello.c</span> that prints the version that has compiled into its executable. Both CMake and GNU Make are supported. Support for BSD Make is missing, so on BSD and Solaris you will have to call <span style="font-family: "courier new" , "courier" , monospace;">gmake </span>instead of <span style="font-family: "courier new" , "courier" , monospace;">make</span>.</div>
<div>
<br /></div>
<div>
First, we start by cloning the template repository:</div>
<pre><code>> hg clone mkversion.hg myproject </code></pre>
<br />
<div>
Then we build the project with autoconf and make and take a look at what we get:<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">> ./configure<br />checking build system type... x86_64-pc-linux-gnu<br />checking host system type... x86_64-pc-linux-gnu<br />checking target system type... x86_64-pc-linux-gnu<br />checking for cc... cc<br />checking whether the C compiler works... yes<br />checking for C compiler default output file name... a.out<br />checking for suffix of executables...<br />checking whether we are cross compiling... no<br />checking for suffix of object files... o<br />checking whether we are using the GNU C compiler... yes<br />checking whether cc accepts -g... yes<br />checking for cc option to accept ISO C89... none needed<br />configure: creating ./config.status<br />config.status: creating Makefile<br />> make<br />cc -MM -MG hello.c -o .depend<br />sh mkversion.sh<br />creating version.h<br />cc -g -O2 -c -o hello.o hello.c<br />cc -g -O2 hello.o -o hello<br />> cat version.h<br />#ifndef VERSION_H<br />#define VERSION_H<br />#define VERSION "V0.1.1 (hg:1/7906498bc6e3)"<br />#define HG_REV "1"<br />#define HG_BRANCH "default"<br />#define HG_NODE "7906498bc6e36f95daf03ffce97a18c3000990fb"<br />#define HG_ID "7906498bc6e3"<br />#define HG_TAGS "tip"<br />#define HG_LATESTTAG "V0.1"<br /><br />#endif<br />> ./hello<br />version V0.1.1 (hg:1/7906498bc6e3)</span></blockquote>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<br />
What we see here, is that we got Version V0.1.1 after cloning the repository. But the latest tag is V0.1. So, let's take a look at the log:<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">> hg log<br />changeset: 1:7906498bc6e3<br />tag: tip<br />user: Thomas Maier-Komor <thomas@maier-komor.de><br />date: Thu Jul 25 07:37:32 2019 +0200<br />summary: Added tag V0.1 for changeset c6295b293642</span> </blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">changeset: 0:c6295b293642<br />tag: V0.1<br />user: Thomas Maier-Komor <thomas@maier-komor.de><br />date: Thu Jul 25 07:37:25 2019 +0200<br />summary: initial checkin of build template with version generation</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">> hg id<br />7906498bc6e3 tip</span></blockquote>
</div>
<div>
As you can see, the clone updated the sandbox to the latest revision, which is the addition of the tag for changeset 0. Therefore, if we want to get the expected version information, we must update to the revision that we tested and tagged afterwards. I.e.:</div>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">> hg up -r V0.1</span></blockquote>
<div>
After that and rebuilding the binary, we get the expected result.</div>
<div>
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">> ./hello<br />version V0.1.0 (hg:0/c6295b293642)</span></blockquote>
</div>
<div>
As written above, Mercurial also provides the infrastructure to determine if the sandbox for building has any uncommitted changes. Like this it is easily possible to integrate this important information into the version string. This template adds a plus character at the end of the version string, if there are uncommitted changes detected at build time. Of course you can change the plus character to something different or even cancel the build if you want to make sure that only reproducible binaries are created. This kind of restriction could also be applied to a specific server.<br />
<br />
Let's see how it works. Just make a simple modification to one of the files that are tracked in the repository. E.g. add a newline at the end of hello.c:<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">> echo >> hello.c</span></blockquote>
After that trigger a new build with make. The version string then looks like this:<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">% ./hello </span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">version V0.1.1+ (hg:1/7906498bc6e3)</span></blockquote>
<div>
This template has no big magic, just a shell script and its integration in the build infrastructure with GNU make and CMake. You can also easily expand it to include the username and/or hostname of the person who triggered the build or whatever else you would like to see. </div>
<div>
<br /></div>
<div>
Get the template as a Mercurial repository <a href="http://www.maier-komor.de/software/mkversion.hg" target="_blank">here</a>. I hope you like it. I am rolling this concept out to all of my software development projects.</div>
</div>
Thomas Maier-Komorhttp://www.blogger.com/profile/10885556724411617546noreply@blogger.com0tag:blogger.com,1999:blog-26028491.post-69747038410744979552019-06-13T01:49:00.002-07:002019-06-13T01:49:25.411-07:00Embedded Systems' Data Serialization<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">Data Serialization</span></h2>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Data serialization is the process of transferring data structures from the in-memory representation that is used for working with the data (e.g. structs) to a binary data stream (usually a sequence of binary octets) that can be transmitted via network or stored on non-volatile memory (e.g. disk, flash). The process in the opposite direction from disk to in memory is called either deserialization or parsing. </span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">There are many ways to serialize data, each having individual advantages and drawbacks. In this post I want to focus on handling generic structured data. Most popular for handling structured data are probably XML and JSON, but there exist also more advanced techniques for creating highly efficient binary data streams.</span></div>
<div>
<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">XML and JSON</span></h2>
</div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><a href="https://en.wikipedia.org/wiki/XML" target="_blank">XML</a> and <a href="https://en.wikipedia.org/wiki/JSON" target="_blank">JSON</a> are both generic file formats that can be used for almost any kind of application and structured data except for <a href="https://en.wikipedia.org/wiki/Binary_large_object" target="_blank">BLOB</a>s (binary large objects - e.g. pictures, video streams, audio data). Both can be used over a variety of platforms, and can be extended easily later if necessary (e.g. later software version with new features), without loosing backward compatibility. Additionally, their representation is human readable, and therefore, it is easy to deal with those kinds of files.</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Comparing XML to JSON, XML provides concepts for meta-data, and the ability to restrict data values to certain ranges to provide means for verification and validation of data. But these features are rather heavy weighted when it comes to their implementation and run-time impact. Nevertheless, powerful embedded systems like smartphones often rely on this technique.</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">For less powerful embedded systems XML is mostly out of scope, as RAM is limited and processing power may be too limited to provide good enough performance. JSON has much lower requirements when it comes to processing power, but it lacks some of those features, which make XML interesting. Nevertheless, JSON libraries for embedded systems are widely available, and JSON is also useful when it comes to interacting with web servers and other generic data processing partners.</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Still, even JSON requires a good amount of code and processing time in comparison with binary file formats. </span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<br />
<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">Google protocol buffers</span></h2>
</div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">There are many binary file formats available for all kinds of applications: images, videos, audio, archives, and many others to name just a few. They use different concepts to encode and compress the data. Providing data serialization for generic data structures is a quite complex task. Therefore, Google provides <a href="https://developers.google.com/protocol-buffers/" target="_blank">Protocol Buffers</a> for specifying structured data and generating code for parsing and serializing data.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Google protocol buffer's language has concepts for extending the file format later. Therefore, every individual data field gets a unique identifier that must not be changed over the life-time of the file format. But the associated data type may be changed and extended when following certain guide-lines. This concepts makes it perfect for use with applications that may need later extensions and may want to obsolete certain functions without breaking compatibility.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Unfortunately, even their lite library implementation is pretty heavy weighted and not targeted for embedded systems. Therefore, I have implemented <a href="http://www.maier-komor.de/wfc.html" target="_blank">Wire Format Compiler</a>, which extends the language of protocol buffers with options and concepts for embedded systems. It is also hosted on <a href="https://github.com/maierkomor/wfc" target="_blank">github</a>, where everybody can participate in the work and file bug reports.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">Wire Format Compiler</span></h2>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Wire Format Compiler's language has been directly derived from protocol buffers. Of course if WFC extensions and features are used, the protocol description will not work with protocol buffers. But it is perfectly possible to come up with data structure specifications that work with both compilers.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">In contrast to protocol buffers, wire format compiler is designed with a strong focus on embedded systems. Therefore, many optimizations for reducing code size and increasing processing performance are implemented to support embedded systems well. Additionally new data types have been added (e.g. </span><span style="font-family: Courier New, Courier, monospace;">fixed8</span><span style="font-family: "trebuchet ms" , sans-serif;">, </span><span style="font-family: Courier New, Courier, monospace;">sfixed16</span><span style="font-family: "trebuchet ms" , sans-serif;">) that support very small controller families, so that targeting even 8- and 16-bit controller families becomes feasible.</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Furthermore, there exists a new concept for specifying options to tailor and optimize one data structure definition for multiple applications on different targets. The serialized data will be usable among all applications, but individual tuning allows to remove unsupported and unneeded data structures by telling the compiler with "used=false" that a specific member will not be used. Furthermore, data types the will be used for strings and byte arrays can be adjusted, so that specialized classes can be used.</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Additionally, many options are provided to generate even more optimized code for specific targets. E.g. there exists support to make optimize for little endian systems that support unaligned access. On the other hand, it is still possible to target also big endian systems that work totally different, when it comes to byte placements in memory.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">A small example</span></h2>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">This is a small example that shows a wfc description and its generated header file. The first part of the wfc description has 3 different option sets that I describe below in more detail. After the option descriptions, a message description follows. This is a data structure that will be generated as a C++ class with methods for handling the data fields, and methods for (de-)serialization. </span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">The description of a field requires three elements: a field type, the field name, and a unique identifier. The unique identifier is set once for every field name, and it must not change later. The associated data is serialized with this unique identifier, and therefore, it is needed for deserialization. Changing the type is only possible, if it is a compatible change. E.g. a variable length integer might be changed from 8 to 16 bits, but the change of type must not impact the serialization concept of the member. Changing the name does only conflict with associated source code. So this is usually without a problem, unless special concepts like JSON support or ASCII generation are used.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-RshClQRV4bk/XQAM-jhL4YI/AAAAAAAAACc/MONPQLlm1Bwp_X2xbFlEWx6axmyptNq1wCLcBGAs/s1600/syscfg_wfc.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="626" data-original-width="731" height="342" src="https://1.bp.blogspot.com/-RshClQRV4bk/XQAM-jhL4YI/AAAAAAAAACc/MONPQLlm1Bwp_X2xbFlEWx6axmyptNq1wCLcBGAs/s400/syscfg_wfc.png" width="400" /></a></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">The first option set, called "</span><span style="font-family: Courier New, Courier, monospace;">embedded</span><span style="font-family: "trebuchet ms" , sans-serif;">", is optimized for a 32-bit embedded systems without string handling. I.e. the strings that are referenced in the data structure are regular C-strings (char pointer), and their memory management must be done independently. In contrast the option set "</span><span style="font-family: Courier New, Courier, monospace;">embedded_dyn</span><span style="font-family: "trebuchet ms" , sans-serif;">" specifies "</span><span style="font-family: Courier New, Courier, monospace;">astring</span><span style="font-family: "trebuchet ms" , sans-serif;">" as string data type. You could also specify your own data type, as long as it has some basic interfaces of </span><span style="font-family: Courier New, Courier, monospace;">std::string</span><span style="font-family: "trebuchet ms" , sans-serif;">, like the member functions </span><span style="font-family: Courier New, Courier, monospace;">size</span><span style="font-family: "trebuchet ms" , sans-serif;">() and </span><span style="font-family: Courier New, Courier, monospace;">c_str</span><span style="font-family: "trebuchet ms" , sans-serif;">(). </span></div>
<div>
<br /></div>
<div>
The generated header file includes a class definition that reflects the message description. It includes a function for calculating the concrete size of a serialized object (<span style="font-family: Courier New, Courier, monospace;">calcSize</span>). This can be used to allocate enough memory before using the <span style="font-family: Courier New, Courier, monospace;">toMemory </span>function to serialize the object to memory. After that the memory block can be written to a file, to flash memory, or send via network, as needed. For deserializing the data on the receiver side the <span style="font-family: Courier New, Courier, monospace;">fromMemory </span>function can be used.</div>
<div>
<br /></div>
<div>
In addition, wfc supports the generation of functions for using a string instead of a memory block for serialization. This is more convenient in use, but might not be possible on all embedded systems. </div>
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-je4IN6rVJDA/XQAM-kIaSZI/AAAAAAAAACg/MDho4CdFHKkP-A5khjNsuu8EOpw3iN7HwCLcBGAs/s1600/syscfg_h_wfc.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="578" data-original-width="731" height="316" src="https://1.bp.blogspot.com/-je4IN6rVJDA/XQAM-kIaSZI/AAAAAAAAACg/MDho4CdFHKkP-A5khjNsuu8EOpw3iN7HwCLcBGAs/s400/syscfg_h_wfc.png" width="400" /></a></div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">For every message object member, functions are generated to get and set their values and to determine the number of elements (size) of repeated members. For optional members functions are added to determine if a certain member has been set (e.g. </span><span style="font-family: Courier New, Courier, monospace;">has_hostname</span><span style="font-family: "trebuchet ms" , sans-serif;">). All generated functions employ statically linked core functions. But options can be used to share the core functionality among multiple message objects.</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">The options and concepts shown in this post are only the most basic ones. Please refer to the available options and documentation, to learn in what other ways the code generation can be influenced. </span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">If you have questions or want to file a bug report, please refer to the <a href="https://github.com/maierkomor/wfc" target="_blank">project page on github</a> or get in touch with me directly.</span></div>
Thomas Maier-Komorhttp://www.blogger.com/profile/10885556724411617546noreply@blogger.com0tag:blogger.com,1999:blog-26028491.post-72396928294757694052019-05-05T14:12:00.004-07:002019-05-19T23:08:44.209-07:00Variants of D1 Mini with ESP8266<h2>
Overview</h2>
In this post I take a look at prototype boards for MCUs from <a href="https://www.blogger.com/http:://www.espressif.com" target="_blank">Espressif</a>. Espressif offers two popular micro-controller families: <a href="https://www.espressif.com/en/products/hardware/esp8266ex/overview" target="_blank">ESP8266</a> and <a href="https://www.espressif.com/en/products/hardware/esp32/overview" target="_blank">ESP32</a>. Those two SOCs have a lot in common, while
the ESP32 is the stronger one with 2 main cores, higher performance, and more peripheral interfaces.<br />
<br />
On the other hand the ESP8266 has lower power consumption and provides sufficient
performance for many applications. Additionally, the ESP8266 family has a member called ESP8285 that provides internal flash memory, so that it can be used
for very small boards, as no additional external flash is necessary.<br />
<br />
Here is a short comparison table of these controllers:
<br />
<br />
<table>
<tbody>
<tr><th>Feature</th><th>ESP8266</th><th>ESP32</th></tr>
<tr><td>architecture</td><td>Tensilica L106</td><td>Tensilica LX6</td></tr>
<tr><td># main cores</td><td>1</td><td>2</td>
</tr>
<tr><td>WiFi MAC</td><td>yes</td><td>yes</td></tr>
<tr><td>Bluetooth MAC</td><td>no</td><td>yes</td></tr>
<tr><td>Ethernet MAC</td><td>no</td><td>yes</td></tr>
<tr><td>touch</td><td>no</td><td>yes</td></tr>
<tr><td>hall sensor</td><td>no</td><td>yes</td></tr>
<tr><td>SPI</td><td>2</td><td>4</td></tr>
<tr><td>PWM</td><td>software</td><td>hardware</td></tr>
<tr><td>waveform generator</td><td>no</td><td>yes</td></tr>
<tr>
</tr>
</tbody></table>
<br />
Both families provide much more features than shown in this short table. This table should give you just an impression of those architectures, in case you
are not familiar with them.<br />
<br />
One popular prototyping board with ESP8266 is the D1 mini offered by <a href="http://www.wemos.cc/" target="_blank">Wemos</a>, which is also provide as a clone by other companies. The variants and versions of the D1 mini have subtle differences, we want to take a look at here.<br />
<br />
<h2>
Boards with ESP8266</h2>
<h3>
D1 Mini:</h3>
<br />
<ul>
<li>ESP8266 </li>
<li>4 MB external flash integrated in metal package</li>
<li>USB socket connect to UART0 TX/RX</li>
<li>Reset button </li>
<li>From Version 2.2.0 to version 2.3.0 the USB socket moved from back to front </li>
<li>one LED </li>
</ul>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-ZUOW2kLaPAo/XM9Nr4KGmkI/AAAAAAAAACI/I7EJa2pX0DcCQkdrVHpCrpulNUaWyQlgQCLcBGAs/s1600/d1minis.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1012" data-original-width="1600" height="404" src="https://1.bp.blogspot.com/-ZUOW2kLaPAo/XM9Nr4KGmkI/AAAAAAAAACI/I7EJa2pX0DcCQkdrVHpCrpulNUaWyQlgQCLcBGAs/s640/d1minis.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-Lf9Jmwdn68g/XM8T7mpzMDI/AAAAAAAAABQ/h3PlSIyKfnY1CC_5qMNOeWfcb1qTQwA-ACLcBGAs/s1600/d1_mini_b.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="881" data-original-width="752" height="320" src="https://1.bp.blogspot.com/-Lf9Jmwdn68g/XM8T7mpzMDI/AAAAAAAAABQ/h3PlSIyKfnY1CC_5qMNOeWfcb1qTQwA-ACLcBGAs/s320/d1_mini_b.png" width="273" /></a></div>
<br />
<br />
<br />
<h3>
D1 Mini Lite:</h3>
<ul>
<li>ESP8285</li>
<li>1MB internal flash</li>
<li>USB socket connect to UART0 TX/RX </li>
<li>Reset button</li>
<li>one LED</li>
</ul>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-9aFnHXZ0IS4/XM8T_6Q7LtI/AAAAAAAAABY/Uj2k9iy6jSQU-j2mom9DyzJjHZbVcZGjQCLcBGAs/s1600/d1_lite_f.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1046" data-original-width="718" height="320" src="https://1.bp.blogspot.com/-9aFnHXZ0IS4/XM8T_6Q7LtI/AAAAAAAAABY/Uj2k9iy6jSQU-j2mom9DyzJjHZbVcZGjQCLcBGAs/s320/d1_lite_f.png" width="219" /></a><a href="https://3.bp.blogspot.com/-u7JDacRDxT4/XM8T98Pbi2I/AAAAAAAAABU/85lwGfcPNksNsOlJUxvD82zimEv9xzgAACLcBGAs/s1600/d1_lite_b.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="998" data-original-width="749" height="320" src="https://3.bp.blogspot.com/-u7JDacRDxT4/XM8T98Pbi2I/AAAAAAAAABU/85lwGfcPNksNsOlJUxvD82zimEv9xzgAACLcBGAs/s320/d1_lite_b.png" width="240" /></a></div>
<br />
<br />
<h3>
Mini32:</h3>
<ul>
<li>Controller: ESP32</li>
<li>Flash: 4MB</li>
<li><strike>USB socket is connected to UART TX </strike><span style="color: red;"><strike>only</strike><br />(Update: this observation is probably related to a broken device. A sample of a similar looking clone without branding did not show this limitation. The clone had an additional LED, so it has not the exact schematic. But the additional LED is just a supply monitoring LED.)</span></li>
<li>Reset button</li>
<li>Pads have same spacing, but are assigned differntly. <span style="color: red;">The critical pads are assigned the same: </span><span style="color: lime;"><span style="color: red;">VCC, GND, TX, RX, 3,3V, RST</span> </span></li>
<li><span style="color: red;"><span style="color: black;">2 LEDs</span></span></li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
</div>
<ul style="margin-left: 1em; margin-right: 1em;"><a href="https://2.bp.blogspot.com/-UMNL7nKf9ks/XM8X7P4cGdI/AAAAAAAAAB8/cSjgSYxxkuYIdoYtYqCZWpeTaG-uASmWgCLcBGAs/s1600/d1_mini_esp32_flash.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><a href="https://1.bp.blogspot.com/-G5hjb-0bRmA/XM8XWet0OFI/AAAAAAAAABw/BAi6e1mQgFY9VoloPWYQ3uRZI-MgDpUWgCLcBGAs/s1600/d1_mini_esp32_b.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><img border="0" data-original-height="893" data-original-width="881" height="400" src="https://1.bp.blogspot.com/-G5hjb-0bRmA/XM8XWet0OFI/AAAAAAAAABw/BAi6e1mQgFY9VoloPWYQ3uRZI-MgDpUWgCLcBGAs/s400/d1_mini_esp32_b.png" width="390" /></ul>
<ul style="margin-left: 1em; margin-right: 1em;"></ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-36A4mcm-eb4/XM8XWaDjPwI/AAAAAAAAABs/IwsrxZUOUoos20iKB2YzM6DCyNXMfDvLACLcBGAs/s1600/d1_mini_esp32_f.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="888" data-original-width="826" height="320" src="https://1.bp.blogspot.com/-36A4mcm-eb4/XM8XWaDjPwI/AAAAAAAAABs/IwsrxZUOUoos20iKB2YzM6DCyNXMfDvLACLcBGAs/s320/d1_mini_esp32_f.png" width="296" /></a></div>
<ul style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="462" data-original-width="734" height="250" src="https://2.bp.blogspot.com/-UMNL7nKf9ks/XM8X7P4cGdI/AAAAAAAAAB8/cSjgSYxxkuYIdoYtYqCZWpeTaG-uASmWgCLcBGAs/s400/d1_mini_esp32_flash.png" width="400" /></ul>
Thomas Maier-Komorhttp://www.blogger.com/profile/10885556724411617546noreply@blogger.com0tag:blogger.com,1999:blog-26028491.post-47661900751143767142019-03-12T16:12:00.001-07:002019-03-12T16:12:42.657-07:00Entscheidung für eine Hausautomatisierung<h2>
Einsatzbereich und Erweiterbarkeit</h2>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Hausautomatisierungen bieten heute ein großes Portfolio an Anwendungsmöglichkeiten. Nicht alle Anbieter von Smart-Home Lösungen bieten Komponenten für alle diese Anwendungen. Deswegen macht es Sinn, sich mit der Frage auseinanderzusetzen, welche Anwendungen in Zukunft in Betracht kommen könnten. </span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Hat man sich einmal für einen Systemanbieter entschieden, muss man mit den Komponenten auskommen, die dieser im Programm hat oder man muss ggf. Systeme verschiedener Anbieter parallel betreiben. Da die Systeme normalerweise zueinander inkompatibel sind, bedeutet das, dass man mehrere Apps verwenden muss, und keine Interaktion über die Systemgrenzen möglich sind.</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Folgende Anwendungen können über Systeme zur Hausautomatisierung abgedeckt werden:</span></div>
<div>
<ul>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Beschattung mit Rollos und Jalousien</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Heizung</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Beleuchtung, Schalten, Dimmen</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Wetter- und Umweltdatenerfassung</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Bewässerung</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Schließsysteme</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Alarm- und Überwachungsanlagen</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Multimedia</span></li>
</ul>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Die Auswahl am Markt ist sehr groß, aber fast alle Systeme sind zueinander inkompatibel. <a href="http://www.conrad.de/" target="_blank">Conrad</a> hat eine schöne <a href="https://www.conrad.de/content/dam/conrad/conrad-de/b2c/images/ratgeber/navigationscontent/gebaeudetechnik/smart-home/smarthome/2.2.4.9.122_SmartHome-Tabelle_A1quer_Kompakt.pdf" target="_blank">Übersicht</a> über die verfügbaren Systeme und ihre Funktionen.</span></div>
</div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Unterstützung für Sprachsteuerung über Alexa und Co. wird von einigen dieser Systeme mit angeboten. Was aber üblicherweise nicht geht, ist beispielsweise eine schaltbare Steckdose eines Anbieters in die Steuerung eines anderen zu integrieren. Auch können Umweltdaten wie Temperatur und Luftfeuchtigkeit eines Sensors von Anbieter A nicht zur Steuerung der Heizung mit einem Thermostat von Anbieter B verwendet werden.</span></div>
<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">Funk oder Kabel?</span></h2>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Grundsätzlich gibt es Kabel gebundene Systeme und Systeme die über Funk kommunizieren. Kabel gebundene Kommunikation ist im Punkt Stör- und Ausfallsicherheit etwas robuster als Funk basierte Systeme. Allerdings kann eine beschädigte Verkabelung zu einem sehr hohen Aufwand bei der Fehlersuche führen.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Funk basierte Systeme wurden erst später eingeführt und können auch in bestehenden Gebäuden einfach nachgerüstet werden, ohne dass nachträglich aufwendig neue Kabel eingezogen werden müssen. Leider behindern Fußböden und Wände aus Beton die Kommunikation via Funk deutlich. Doch es gibt Zusatzgeräte um in Gebäuden mit viel Beton eine Hausautomatisierung mit Funk in jeder Ecke zu ermöglichen.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Die meisten kabellose Systeme verwenden das 868 MHz Band. Allerdings ist diese Kommunikation nicht standardisiert. Das bedeutet, dass Systeme verschiedener Anbieter die eine 868MHz basierte Lösung haben, normalerweise nicht miteinander kompatibel sind und somit nicht miteinander verwendet werden können. </span><span style="font-family: "trebuchet ms" , sans-serif;">Deswegen sollte man immer davon ausgehen, dass Systeme verschiedener Anbieter nicht miteinander funktionieren. </span></div>
<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">Bedienung und Integration des Designs</span></h2>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Über die Art und Weise der Bedienung sollte man sich im Vorfeld auch Gedanken machen. Ist eine Bedienung per Wand- oder Handschalter gewünscht? Ist ein Tischaufsteller vorzuziehen? Soll der Status der Systeme irgendwie visualisiert werden (z.B. Display an der Wand)? Wie sieht es aus mit einer App für Smartphone oder Tablet - würde diese von den Bewohnern verwendet werden?</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Nicht jeder Anbieter bietet alle Bedienkonzepte, und häufig haben die Anbieter ein eigenes Design. Das kann insbesondere auf Ablehnung stoßen, wenn die Wandschalter zwischen Hausautomatisierung und normalen Schaltern unterschiedlich aussehen. Einige Hersteller bieten eine Integration ins bestehende Schalterprogramm an. Dabei werden die Blenden des Anbieters für das Schalterprogramm über Adapter auf die Komponenten der Hausautomatisierung gesteckt.</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Alternativ kann man das ganze Schalterprogramm tauschen (also alle Lichtschalter und Steckdosen). Allerdings treibt das die Kosten unter Umständen deutlich in die Höhe.</span></div>
<div>
<br /></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Wenn der bestehende Look des Schalterprogramms erhalten werden soll, muss sichergestellt werden, dass es für die konkrete Designvariante passende Adapter gibt. Diese sind oft für die bekannten Hersteller von Schaltern und Steckdosen, wie Busch-Jaeger, Gira, Jung und andere verfügbar. </span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Manche Hersteller von Schalterprogrammen bieten auch eigene Lösungen zum Thema Smart Home an. Dann stellt sich die Frage des passenden Designs gar nicht erst. <a href="https://www.busch-jaeger.de/" target="_blank">Busch-Jaeger</a> und <a href="https://www.gira.de/" target="_blank">Gira</a> haben beispielsweise eine gemeinsame technische Lösung names <a href="https://www.enet-smarthome.com/de/" target="_blank">eNet</a>. Allerdings ist hier zu beachten, dass die klassischen Anbieter von Schalterprogrammen sich auf ihre klassischen Anwendungen konzentrieren und für die Installation von geschultem Fachpersonal ausgehen. </span></div>
<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">Zentralsteuerung vs. Internetbasierter Lösung</span></h2>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Smart Home Lösungen arbeiten entweder mit einem Server der zu Hause aufgestellt wird oder mit einer Cloud basierten Lösung im Internet. </span></div>
<div>
<h3>
<span style="font-family: "trebuchet ms" , sans-serif; font-weight: normal;">Vorteile eine Servers zu Hause:</span></h3>
</div>
<div>
<ul>
<li><span style="font-family: "trebuchet ms" , sans-serif;">funktioniert auch, wenn die Internetverbindung zusammenbricht</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">alle Daten bleiben zu Hause</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Funktionsfähigkeit hängt nur an den beschafften Geräten und nicht an externen Firmenentscheidungen (z.B. Pleite, Abkündigung von Cloud-Diensten)</span></li>
</ul>
<div>
<h3>
<span style="font-family: "trebuchet ms" , sans-serif; font-weight: normal;">Vorteile einer Cloud-Lösung (Server im Internet):</span></h3>
</div>
</div>
<div>
<ul>
<li><span style="font-family: "trebuchet ms" , sans-serif;">geringere Anschaffungskosten</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">geringerer Installationsaufwand</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">der Zugriff über das Internet in der Regel ohne Mehraufwand möglich</span></li>
</ul>
</div>
<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">Entscheidung für ein System</span></h2>
<h3>
<span style="font-family: "trebuchet ms" , sans-serif; font-weight: normal;">Meine Anforderungen:</span></h3>
<div>
<ul>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Integration ins bestehende Schaltersystem Gira E2</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Funkbasierte Lösung, da an vielen Stellen nachträglich keine Verkabelung möglich ist</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Unterstützung von Jalousien und dem Schrägstellen der Jalousielamellen für eine Teilbeschattung</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Abdeckung möglichst vieler Anwendungsgebiete</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Optional ein offenes System, so dass auch Eigenlösungen integriert werden können</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">App für die einfache Bedienung wäre von Vorteil</span></li>
</ul>
<h3>
<span style="font-family: "trebuchet ms" , sans-serif; font-weight: normal;">Lösungsfindung:</span></h3>
</div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Unser Schalterprogramm ist von Gira, deswegen habe ich als erstes das Angebot von Gira evaluiert. Gira bietet alles was ich zu Beginn haben wollte. Einzig störte mich, dass eNet ein geschlossenes System ist, in das keine Eigenlösungen integriert werden kann. Der Preis für den eNet Server bewegt sich im Mittelfeld und im Vergleich zur kabelgebundenen KNX Lösungen ist er deutlich günstiger. Außerdem sind die angebotenen Komponenten auf die klassischen Anwendungen eines Schalterprogramms beschränkt.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Lösungen mit dem Gira E2 Design gibt es auch von anderen Herstellern. Am Ende habe ich mich für <a href="https://www.homematic-ip.com/start.html" target="_blank">Homematic IP</a> entschieden, da dies als Server Lösung mit CCU3 kompatibel zu <a href="https://www.homematic.com/" target="_blank">Homematic</a> ist und außerdem als Cloud-Variante einen einfachen und sehr kostengünstigen Einstieg mit vielen Erweiterungsmöglichkeiten bietet. Außerdem ist das Homematic-System teiloffen und es gibt Möglichkeiten Eigenlösungen zu entwickeln. Darüber hinaus werden Alexa, Google Assistant und Conrad Connect auch noch unterstützt.</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Die Konkurrenzprodukte von Bosch, Innogy und Co. habe ich wegen fehlenden Anwendungsmöglichkeiten ausgeschlossen. Die Auswahl der richtigen Komponenten bei Gira gestaltete sich schwierig, da es mehrere verschiedene Systeme gibt, die nicht alle beliebig miteinander kombiniert werden können.</span></div>
<div>
<h3>
<span style="font-family: "trebuchet ms" , sans-serif; font-weight: normal;">Meine Lösung:</span></h3>
<span style="font-family: "trebuchet ms" , sans-serif;">Zunächst habe ich die Cloud basierte Variante von Homematic-IP mit dem für 50 Euro sehr günstigen AccessPoint in Betrieb genommen. Meine Erwartungshaltung war hier, dass ich mit der App recht schnell an die Grenzen dessen komme was möglich ist und Wünsche offen bleiben. In diesem Fall wäre der Schritt zu einer CCU3 als Home-Server einfach möglich, da alle Geräte von Homematic-IP auch damit kompatibel sind. Anders herum sind leider die Geräte von Homematic nicht mit der Cloud-Lösung von Homematic-IP kompatibel. </span><span style="font-family: "trebuchet ms" , sans-serif;">Die Cloud basierte Lösung bleibt somit auf das Angebot von Homematic-IP beschränkt.</span><br />
<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">Erfahrungen bei Installation und Betrieb</span></h2>
<span style="font-family: "trebuchet ms" , sans-serif;">Die Installation der Hardware benötigt für alle Geräte einen Nullleiter (schwarzer Draht). Das ist auch für alle Konkurenzprodukte so zu erwarten. D.h. man sollte vor dem Kauf der Geräte prüfen, ob die Dosen alle Anforderungen bzgl. des Bauraums und der erforderlichen Leitungen erfüllen. Ggf. muss noch ein Draht eingezogen werden, wenn die entsprechende Dose noch keinen Nullleiter hat. Die Installation sollte auf jedem Fall von jemanden durchgeführt werden, der über das entsprechende elektrische Fachwissen verfügt. </span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<b><span style="font-family: "trebuchet ms" , sans-serif;">B</span><span style="font-family: "trebuchet ms" , sans-serif;">itte immer daran denken, dass zum prüfen des Bauraums und der verfügbaren Leitungen die entsprechenden Phasen über die Sicherungen zum Eigenschutz abgeschaltet werden müssen.</span></b></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Persönlich hat mich bei der Inbetriebnahme überrascht, dass ich mit der App zur Konfiguration des Cloud Dienstes tatsächlich alle meine Wünsche bzgl. der Beschattungskonfiguration mit Gruppensteuerungen, Zeitprofilen und Abhängigkeiten zu Sonnen Auf- und Untergang realisieren konnte. Der Einstieg erfolgte mit 4 Rollladen Steuerungen und 4 Jalousiesteuerungen. Inzwischen sind bereits mehr Geräte integriert und die nächste Erweiterung für Aussperrschutz (Türsensor der Verhindert, dass die zugehörige Jalousie heruntergefahren wird, wenn die Tür offen steht) und die Heizung um Keller bestellt. </span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Sogar ein Schutz der Jalousien in Abhängigkeit von der Windgeschwindigkeit basierend auf den Wetterdaten im Internet zu der Postleitzahl war möglich. Ein richtiger Windmesser einer Wetterstation liefert allerdings genauere Daten und darüber hinaus Informationen zur Sonnensituation.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Die Funk-Reichweite in unserem Haus mit viel Beton ist grenzwertig. Trotzdem war durch eine geeignete Positionierung des Accesspoint im ersten Obergeschoß es bisher möglich alle Geräte zu erreichen. Eine Reichweitenerhöhung über Steckdosenrepeater mit Schaltfunktion ist ggf. für ca. 40 Euro möglich. </span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">In Summe hat diese Lösung weniger als die Hälfte dessen gekostet, was die gleiche Lösung von Gira gekostet hätte. Außerdem ist bereits von Anfang an ein höherer Funktionsumfang durch die Cloud-Technik gegeben (z.B. Windschutz der Jalousien). </span></div>
<div>
<h3>
<span style="font-family: "trebuchet ms" , sans-serif;"><span style="font-weight: normal;">Die App</span></span></h3>
</div>
<span style="font-family: "trebuchet ms" , sans-serif;">Die App von Homematic IP ist tatsächlich auch für einen Endanwender vernünftig nutzbar und zeigt den Status auch unterwegs ohne Zusatzaufwand vernünftig an.</span><br />
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Hier einige beispielhafte Screenshots, wie die App zur Bedienung über das Smartphone aussieht. Im Falle von Homematic-IP erfolgte auch die gesamte Konfiguration und das Update der Komponenten über diese App. Aus meiner Sicht ist die App mit ihrer Aufteilung in Räume, Gruppen und Zeitprofilen sauber und verständlich strukturiert.</span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-W6peNXR4l0Q/XIe2J91zooI/AAAAAAAAAAM/qvNzGKR-L8kxKdA3CYnhe81pRE1n7El0ACLcBGAs/s1600/Screenshot_2019-03-09-11-49-27.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="900" height="320" src="https://2.bp.blogspot.com/-W6peNXR4l0Q/XIe2J91zooI/AAAAAAAAAAM/qvNzGKR-L8kxKdA3CYnhe81pRE1n7El0ACLcBGAs/s320/Screenshot_2019-03-09-11-49-27.png" width="180" /></a></div>
<a href="https://2.bp.blogspot.com/-L0Ru9Rw-7NM/XIe2NMTafOI/AAAAAAAAAAQ/Y05OuPXh1YwldbprE7GriFWIFFLk6ZQ7ACLcBGAs/s1600/Screenshot_2019-03-09-11-49-38.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1600" data-original-width="900" height="320" src="https://2.bp.blogspot.com/-L0Ru9Rw-7NM/XIe2NMTafOI/AAAAAAAAAAQ/Y05OuPXh1YwldbprE7GriFWIFFLk6ZQ7ACLcBGAs/s320/Screenshot_2019-03-09-11-49-38.png" width="180" /></a> <a href="https://2.bp.blogspot.com/-a8GUqd0dVsw/XIe2NMogV4I/AAAAAAAAAAU/W1NtPxuqlSoZWtfDOa1FSfko5nIlOobHACLcBGAs/s1600/Screenshot_2019-03-09-11-50-19.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1600" data-original-width="900" height="320" src="https://2.bp.blogspot.com/-a8GUqd0dVsw/XIe2NMogV4I/AAAAAAAAAAU/W1NtPxuqlSoZWtfDOa1FSfko5nIlOobHACLcBGAs/s320/Screenshot_2019-03-09-11-50-19.png" width="180" /></a><br />
<div>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Power-User kommen mit der App möglicherweise an die Grenze dessen, was die App kann. In diesem Fall kann man bei Homematic-IP aber immer noch auf einen Server (CCU3) umstellen, der dann eine deutlich detailliertere und aufwendigere Konfiguration ermöglicht. Bei der Verwendung der CCU3 können dann auch noch Komponenten aus dem Angebot von Homematic (also nicht Homematic-IP) verwendet werden.</span></div>
<h2>
<span style="font-family: "trebuchet ms" , sans-serif;">Fazit</span></h2>
<div>
<span style="font-family: "trebuchet ms" , sans-serif;">Homematic-IP als Cloud basierte Smart-Home-Lösung kann ich durchaus empfehlen. Mangels Erfahrung mit Konkurrenzprodukten ist allerdings eine differenzierte Bewertung an dieser Stelle nicht möglich. So bleibt nur mein </span><span style="font-family: "trebuchet ms" , sans-serif;">subjektiver aber </span><span style="font-family: "trebuchet ms" , sans-serif;">positiver Gesamteindruck. Außerdem sehe ich auch im Nachhinein am Markt keine bessere Lösung und würde es so auch wieder umsetzen.</span></div>
<div>
<h3>
<span style="font-family: "trebuchet ms" , sans-serif; font-weight: normal;">Positiv ist mir aufgefallen:</span></h3>
</div>
<div>
<ul>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Inbetriebnahme des Accesspoints</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Verständlichkeit und Aufwand der Konfiguration</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Einfache Bedienung über App und Taster</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Verwendung von Windinformationen auf Postleitzahlbasis zum Schutz der Jalousien</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Steuerung in Abhängigkeit von Sonnenauf- und -untergang</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Automatische Erkennung der Abschaltpunkte und Laufzeiten von Rollos und Jalousien</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Automatisches Update der Geräte</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">Preis/Leistungs-Verhältnis</span></li>
</ul>
<div>
<h3>
<span style="font-weight: normal;"><span style="font-family: "trebuchet ms" , sans-serif;">Verbesserungswürdig</span><span style="font-family: "trebuchet ms" , sans-serif;"> </span><span style="font-family: "trebuchet ms" , sans-serif;">habe ich empfunden:</span></span></h3>
</div>
</div>
<div>
<ul>
<li><span style="font-family: "trebuchet ms" , sans-serif;">minimale Steuerzeit der Jalousien erlaubt keine Feinpositionierung der Lamellenwinkel (vermutlich Systembedingt wegen der verbauten Relais)</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">schwacher Druckpunkt der Steuertaster</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">manche Änderungen waren nur über ein Abmelden, Rücksetzen und Wiederanmelden der Geräte möglich</span></li>
<li><span style="font-family: "trebuchet ms" , sans-serif;">das Verschraubungs- und Befestigungskonzept erlaubt nur begrenzte Feinpositionierung</span></li>
</ul>
</div>
Thomas Maier-Komorhttp://www.blogger.com/profile/10885556724411617546noreply@blogger.com0München, Deutschland48.1351253 11.58198049999998647.965637799999996 11.259256999999986 48.3046128 11.904703999999986tag:blogger.com,1999:blog-26028491.post-85684630851252065952019-03-05T03:59:00.001-08:002019-03-05T15:35:37.430-08:00Speedport Smart 3 vs. FritzBox - Erfahrungen...Nach ein paar Jahren mit einem anderen Internet-Anbieter bin ich nun zur Telekom gewechselt. Deswegen stand die Frage im Raum: <a href="https://www.telekom.de/zuhause/geraete-und-zubehoer/wlan-und-router/speedport-smart-3" target="_blank">Speedport Smart 3 </a>mieten oder ein eigenes Gerät kaufen. Da die Mietkonditionen für Neukunden sehr günstig sind, und das WLAN To Go nur mit dem Speedport angeboten wird, habe ich mich zunächst einmal für die Miete entschieden.<br />
<br />
Eigentlich bin ich davon ausgegangen, dass es technisch keinen Unterschied machen sollte, ob eine <a href="https://avm.de/produkte/fritzbox/" target="_blank">FritzBox</a> (hatte ich zuvor) als Router verwendet wird oder ein Speedport Smart 3 von der Telekom. Leider stimmt das nicht bei meinen Anforderungen. Denn es gibt mit dem Speedport einiges worauf man verzichten muss:<br />
<ol>
<li>Die Liste der verpassten Anrufe und das Telefonbuch werden vom Speedport nicht auf dem AVM Telefon unterstützt, und das Wählverfahren läuft nur im Ton-Wahlverfahren. </li>
<li>VPN ist nicht im Speedport direkt konfigurierbar und nur über eine Portfreischaltung realisierbar. D.h. ein zusätzliches Gerät muss als VPN Server konfiguriert werden (z.B. ein <a href="https://www.raspberrypi.org/" target="_blank">RasPi</a> mit <a href="http://www.pivpn.io/" target="_blank">Pi-VPN</a>). Durch das zusätzliches Gerät sind die Stromkosten höher, und das eigenhändige Pflegen eines VPN Servers bedeutet Mehraufwand bietet bringt zusätzliche Sicherheitsrisiken mit sich.</li>
<li>Der DHCP Server des Speedport gibt die Namenseinträge der Gerät nicht an den Nameserver weiter. D.h. die Geräte sind dadurch nicht wie von der FritzBox gewohnt über ihren Hostnamen sondern nur über ihre IP-Adresse ansprechbar. Das hat bei mir alle <a href="https://owncloud.org/" target="_blank">Owncloud</a> Dienste auf den Clients lahmgelegt und ist auch sonst ein nerviges Hindernis für Netzwerkdienste wie z.B. SSH, NFS. </li>
<li>Es gibt keinen Gast-LAN Port. Diesen hatte ich bisher dafür benutzt unsere auf <a href="https://www.homematic-ip.com/start.html" target="_blank">Homematic IP</a> basierende Hausautomatisierung vom restlichen Netz zu isolieren. Das ist nicht notwendig, fand ich aber ein nützliches Detail, da so eine Sicherheitslücke im Homematic-IP Gerät sich nicht nutzen lässt, um auf das Heimnetz zuzugreifen.</li>
<li>WLAN-Mesh: die <a href="https://avm.de/produkte/fritzwlan/" target="_blank">AVM Repeater</a> lassen sich nicht mit dem Speedport n ein gemeinsam konfigurierbares und Update-bares Mesh zusammenführen. Also auch hier Mehraufwand.</li>
</ol>
Entsprechend habe ich kurzer Hand eine <a href="https://avm.de/produkte/fritzbox/fritzbox-7530/">FritzBox 7530</a> gekauft und werde die Miete kündigen. Einziges Manko mit der 7530: Das Gast-LAN (nicht Gast-WLAN) ist bei Verwendung der 7530 an einem Glasfaseranschluss nicht Verfügbar, da anscheinend das Glasfasermodem auf dem Ethernet-Port des Gast-LAN angeschlossen wird...<br />
<br />
Update: Das Gast-LAN der FritzBox kann nach der Einstellung der Glasfaser-Verbindung wieder reaktiviert werden.Thomas Maier-Komorhttp://www.blogger.com/profile/10885556724411617546noreply@blogger.com0