background preloader

Data Serialization

Facebook Twitter

Présentation du protocole Syslog. Le protocole Syslog est défini par les RFC suivantes: Le protocole Syslog est un protocole en mode "texte", c'est-à-dire qu'il utilise uniquement les caractères du code ASCII.

Présentation du protocole Syslog

Il utilise le protocole UDP et le port 514 mais il faut savoir qu'il existe aussi des implémentations de Syslog en TCP ou même en SSL et sur d'autres numéros de port. La longueur totale d'une trame Syslog doit être de 1024 octets ou moins. Il n'y a pas de longueur minimale mais une trame de 0 octet n'est pas très utile et ne devrait pas être transmise. Une trame de protocole Syslog est composée de 3 parties : Google Protocol Buffers vs Apache Avro.

I have been looking into middleware solutions as a push mechanism between server and client.

Google Protocol Buffers vs Apache Avro

One of the aspects that I had to consider was latency. Hence it was important to find a lean technology agnostic transport format. Most server-client platforms use a serialization technique to serialize into a leaner data format, and then de-serialize on the receiving end. So in my search for a technology I now had to consider speed of serialization. Many languages offer native serialization APIs, but when serializing the data using the native API, Metadata about the class is serialized into the output too. I also needed to identify the best data format to serialize to.

I came across two technologies ‘Google Protocol Buffers’ and ‘Apache Avro’: Google Protocol Buffers Protocol Buffers is a serialisation format with an interface description language developed by Google. It works by you defining how you want your data to be structured via proto files, which are simply structure text files. Results. Protobuf - Protocol Buffers - Google's data interchange format. What is it?

protobuf - Protocol Buffers - Google's data interchange format

Protocol Buffers are a way of encoding structured data in an efficient yet extensible format. Google uses Protocol Buffers for almost all of its internal RPC protocols and file formats. Latest Updates Documentation Read the documentation. Discussion Visit the discussion group. Quick Example You write a .proto file like this: message Person { required int32 id = 1; required string name = 2; optional string email = 3;} Then you compile it with protoc, the protocol buffer compiler, to produce code in C++, Java, or Python.

Then, if you are using C++, you use that code like this: Person person;person.set_id(123);person.set_name("Bob");person.set_email("bob@example.com"); fstream out("person.pb", ios::out | ios::binary | ios::trunc);person.SerializeToOstream(&out);out.close(); Or like this: Person person;fstream in("person.pb", ios::in | ios::binary);if (! Thrift vs Protocol Bufffers vs JSON. Note: This article a work in progress.

Thrift vs Protocol Bufffers vs JSON

If there is anything that needs correcting please let me know by leaving a comment. Originally this comparison included a look at JSON. Although JSON is small, lightweight, fast to transmit and easily serialized/de-serialized, it is disqualified simply on the basis that there is no built-in way to allow versioning of objects. Both Protobuf and Thrift allow some type of data versioning so that both clients and servers can continue to work without being upgraded, even if the protocol has changed.

This is handy when rolling out a new protocol since there’s no need to orchestrate a massive protocol update across services before flipping the switch on a new protocol. Much of this table was originally compiled by Stuart Sierra but has been edited to include additional information relevant to my own requirements. Thrift and Protocol Buffers are both great choices and there seems like no clear winner between them. I’d choose Protocol Buffers over Thrift if: Apache Thrift. Welcome to Apache Avro!

The Fat-Free Alternative to XML. Extensible Markup Language (XML) is a text format derived from Standard Generalized Markup Language (SGML).

The Fat-Free Alternative to XML

Compared to SGML, XML is simple. HyperText Markup Language (HTML), by comparison, is even simpler. Even so, a good reference book on HTML is an inch thick. This is because the formatting and structuring of documents is a complicated business. Most of the excitement around XML is around a new role as an interchangeable data serialization format. It is text-based. These together encouraged a higher level of application-independence than other data-interchange formats.

Unfortunately, XML is not well suited to data-interchange, much as a wrench is not well-suited to driving nails. The most informed opinions on XML (see for example xmlsuck.org) suggest that XML has big problems as a data-interchange format, but the disadvantages are compensated for by the benefits of interoperability and openness. Let's compare XML and JSON on the attributes that the XML community considers important.