1
0
Fork 0
mirror of git://git.psyc.eu/libpsyc synced 2024-08-15 03:19:02 +00:00

bench: moved packets to separate files, put results in the main document

This commit is contained in:
tg(x) 2011-05-17 12:27:07 +02:00
parent a9e8d94bdd
commit e9f1368fc0
19 changed files with 166 additions and 459 deletions

1
bench/.gitignore vendored
View file

@ -1,3 +1,2 @@
*.html *.html
*.pdf *.pdf
packets/

View file

@ -8,22 +8,7 @@ INIT = (setq load-path (cons \"/usr/share/emacs/site-lisp/org-mode\" load-path)
org-src-preserve-indentation t) \ org-src-preserve-indentation t) \
(require 'org-install) (require 'org-install)
ORG = benchmark.org results.org ORG = benchmark.org
wiki2org:
perl -pe '\
s/^= (.*) =\s*$$/#+TITLE: $$1\n/; \
s/^== (.*) ==\s*$$/* $$1/; \
s/^=== (.*) ===\s*$$/** $$1/; \
s/^{{{/#+BEGIN_SRC/; \
s/^}}}/#+END_SRC/ \
' benchmark.wiki >benchmark.org
packets:
emacs -Q --batch --eval \
"(progn ${INIT} (find-file \"benchmark.org\") \
(org-babel-tangle) (kill-buffer))"
perl -pi -e 'print "\n" unless $$p; $$p=1' packets/user_profile.psyc
html: html:
for f in ${ORG}; do \ for f in ${ORG}; do \

View file

@ -19,30 +19,17 @@ Since presence packets are by far the dominant messaging content
in the XMPP network, we'll start with one of them. in the XMPP network, we'll start with one of them.
Here's an example from paragraph 4.4.2 of RFC 6121. Here's an example from paragraph 4.4.2 of RFC 6121.
#+BEGIN_SRC xml :tangle packets/presence.xml #+INCLUDE: packets/presence.xml src xml
<presence from='juliet@example.com/balcony'
to='benvolio@example.net'>
<show>away</show>
</presence>
#+END_SRC
And here's the same information in a JSON rendition: And here's the same information in a JSON rendition:
#+BEGIN_SRC js :tangle packets/presence.json #INCLUDE: packets/presence.json src js
["presence",{"from":"juliet@example.com/balcony","to":"benvolio@example.net"},{"show":"away"}]
#+END_SRC
Here's the equivalent PSYC packet in verbose form Here's the equivalent PSYC packet in verbose form
(since it is a multicast, the single recipients do not (since it is a multicast, the single recipients do not
need to be mentioned): need to be mentioned):
#+BEGIN_SRC psyc :tangle packets/presence.psyc #+INCLUDE: packets/presence.psyc src psyc
:_context psyc://example.com/~juliet
=_degree_availability 4
_notice_presence
|
#+END_SRC
And the same in compact form: And the same in compact form:
@ -58,133 +45,38 @@ np
XML: XML:
#+BEGIN_SRC xml :tangle packets/chat_msg.xml #+INCLUDE: packets/chat_msg.xml src xml
<message
from='juliet@example.com/balcony'
id='ktx72v49'
to='romeo@example.net'
type='chat'
xml:lang='en'>
<body>Art thou not Romeo, and a Montague?</body>
</message>
#+END_SRC
PSYC: PSYC:
#+BEGIN_SRC psyc :tangle packets/chat_msg.psyc #+INCLUDE: packets/chat_msg.psyc src psyc
:_source psyc://example.com/~juliet
:_target psyc://example.net/~romeo
_message_private
Art thou not Romeo, and a Montague?
|
#+END_SRC
** A new status updated activity ** A new status updated activity
Example taken from http://onesocialweb.org/spec/1.0/osw-activities.html Example taken from http://onesocialweb.org/spec/1.0/osw-activities.html
You could call this XML namespace hell: You could call this XML namespace hell:
#+BEGIN_SRC xml :tangle packets/activity.xml #+INCLUDE: packets/activity.xml src xml
<iq type='set'
from='hamlet@denmark.lit/snsclient'
to='hamlet@denmark.lit'
id='osw1'>
<pubsub xmlns='http://jabber.org/protocol/pubsub'>
<publish node='urn:xmpp:microblog:0'>
<item>
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:activity="http://activitystrea.ms/spec/1.0/"
xmlns:osw="http://onesocialweb.org/spec/1.0/">
<title>to be or not to be ?</title>
<activity:verb>http://activitystrea.ms/schema/1.0/post</activity:verb>
<activity:object>
<activity:object-type>http://onesocialweb.org/spec/1.0/object/status</activity:object-type>
<content type="text/plain">to be or not to be ?</content>
</activity:object>
<osw:acl-rule>
<osw:acl-action permission="http://onesocialweb.org/spec/1.0/acl/permission/grant">
http://onesocialweb.org/spec/1.0/acl/action/view
</osw:acl-action>
<osw:acl-subject type="http://onesocialweb.org/spec/1.0/acl/subject/everyone"/>
</osw:acl-rule>
</entry>
</item>
</publish>
</pubsub>
</iq>
#+END_SRC
http://activitystrea.ms/head/json-activity.html proposes a JSON encoding http://activitystrea.ms/head/json-activity.html proposes a JSON encoding
of this. We'll have to add a routing header to it. of this. We'll have to add a routing header to it.
#+BEGIN_SRC js :tangle packets/activity.json #+INCLUDE: packets/activity.json src js
["activity",{"from":"hamlet@denmark.lit/snsclient"},{"verb":"post",
"title":"to be or not to be ?","object":{"type":"status",
"content":"to be or not to be ?","contentType":"text/plain"}}]
#+END_SRC
http://about.psyc.eu/Activity suggests a PSYC mapping for activity http://about.psyc.eu/Activity suggests a PSYC mapping for activity
streams. Should a "status post" be considered equivalent to a presence streams. Should a "status post" be considered equivalent to a presence
description announcement or just a message in the "microblogging" channel? description announcement or just a message in the "microblogging" channel?
We'll use the latter here: We'll use the latter here:
#+BEGIN_SRC psyc :tangle packets/activity.psyc #+INCLUDE: packets/activity.psyc src psyc
:_context psyc://denmark.lit/~hamlet#_follow
:_subject to be or not to be ?
:_type_content text/plain
_message
to be or not to be ?
|
#+END_SRC
** A message with JSON-unfriendly characters ** A message with JSON-unfriendly characters
#+BEGIN_SRC xml :tangle packets/json-unfriendly.xml #+INCLUDE: packets/json-unfriendly.xml src xml
<message
from='romeo@example.net/orchard'
id='sl3nx51f'
to='juliet@example.com/balcony'
type='chat'
xml:lang='en'>
<body>"Neither, fair saint, if either thee dislike.", he said.
And
the
rest
is
history.</body>
</message>
#+END_SRC
** A message with XML-unfriendly characters ** A message with XML-unfriendly characters
#+BEGIN_SRC xml :tangle packets/xml-unfriendly.xml #+INCLUDE: packets/xml-unfriendly.xml src xml
<message
from='juliet@example.com/balcony'
id='z94nb37h'
to='romeo@example.net'
type='chat'
xml:lang='en'>
<body>Wherefore art thou, Romeo?</body>
<body xml:lang='cs'>
Pro&#x010D;e&#x017D; jsi ty, Romeo?
</body>
</message>
#+END_SRC
** A message with PSYC-unfriendly strings ** A message with PSYC-unfriendly strings
#+BEGIN_SRC xml :tangle packets/psyc-unfriendly.xml #+INCLUDE: packets/psyc-unfriendly.xml src xml
<message
from='juliet@example.com/balcony'
id='c8xg3nf8'
to='romeo@example.net'
type='chat'
xml:lang='en'>
<subject>I implore you with a pointless
newline in a header variable</subject>
<body>Wherefore art thou, Romeo?
|
And for practicing purposes we added a PSYC packet delimiter.</body>
</message>
#+END_SRC
** A packet containing a JPEG photograph ** A packet containing a JPEG photograph
... TBD ... ... TBD ...
@ -194,41 +86,31 @@ In this test we'll not consider XMPP at all and simply compare the
efficiency of the three syntaxes at serializing a typical user data base efficiency of the three syntaxes at serializing a typical user data base
storage information. We'll again start with XML: storage information. We'll again start with XML:
#+BEGIN_SRC xml :tangle packets/user_profile.xml #+INCLUDE: packets/user_profile.xml src xml
<UserProfile>
<Name>Silvio Berlusconi</Name>
<JobTitle>Premier</JobTitle>
<Country>I</Country>
<Address>
<Street>Via del Colosseo, 1</Street>
<PostalCode>00100</PostalCode>
<City>Roma</City>
</Address>
<Page>http://example.org</Page>
</UserProfile>
#+END_SRC
In JSON this would look like this: In JSON this would look like this:
#+BEGIN_SRC js :tangle packets/user_profile.json #+INCLUDE: packets/user_profile.json src js
["UserProfile",{"Name":"Silvio Berlusconi","JobTitle":"Premier","Country":"I","Address":
{"Street":"Via del Colosseo, 1","PostalCode":"00100","City":"Roma"},"Page":"http://example.org"}]
#+END_SRC
Here's a way to model this in PSYC: Here's a way to model this in PSYC:
#+BEGIN_SRC psyc :tangle packets/user_profile.psyc #+INCLUDE: packets/user_profile.psyc src psyc
:_name Silvio Berlusconi * Results
:_title_job Premier
:_country I Parsing time of 1 000 000 packets in milliseconds:
:_address_street Via del Colosseo, 1
:_address_code_postal 00100 | input: | PSYC | | JSON | | | XML | |
:_address_city Roma | parser: | strlen | libpsyc | json-c | json-glib | libxml sax | libxml | rapidxml |
:_page http://example.org |-----------+--------+---------+--------+-----------+------------+--------+----------|
_profile_user | presence | 30 | 246 | 2463 | 10197 | 4997 | 7557 | 1719 |
| | chat msg | 41 | 320 | | | 5997 | 9777 | 1893 |
#+END_SRC | activity | 42 | 366 | 4666 | 16846 | 13357 | 28858 | 4419 |
| user prof | 55 | 608 | 4715 | 17468 | 7350 | 12377 | 2477 |
|-----------+--------+---------+--------+-----------+------------+--------+----------|
| / | < | > | < | > | < | | > |
These tests were performed on a 2.53 GHz Intel(R) Core(TM)2 Duo P9500 CPU.
* Conclusions * Conclusions
... TBD ... ... TBD ...
@ -268,3 +150,19 @@ After a month of development libpsyc is already performing pretty
well, but we presume various optimizations, like rewriting parts well, but we presume various optimizations, like rewriting parts
in assembler, are possible. in assembler, are possible.
* Appendix
** Tools used
libpsyc:
: test/testStrlen -sc 1000000 -f $file
: test/testPsycSpeed -sc 1000000 -f $file
: test/testJson -snc 1000000 -f $file
: test/testJsonGlib -snc 1000000 -f $file
xmlbench:
: parse/libxml-sax 1000000 $file
: parse/libxml 1000000 $file
: parse/rapidxml 1000000 $file

View file

@ -1,269 +0,0 @@
= libpsyc Performance Benchmarks =
In this document we present the results of performance benchmarks
of libpsyc compared with libjson-glib and libxml2.
== Procedure ==
We'll use typical messages from the XMPP ("stanzas" in Jabber
lingo) and compare them with equivalent JSON encodings,
verbose and compact PSYC formats.
In some cases we will additionally compare PSYC packets to
a more efficient XML encoding based on PSYC methods, to have
a more accurate comparison of the actual PSYC and XML
syntaxes, rather than the protocol structures of PSYC and XMPP.
== The Benchmarks ==
=== A presence packet ===
Since presence packets are by far the dominant messaging content
in the XMPP network, we'll start with one of them.
Here's an example from paragraph 4.4.2 of RFC 6121.
{{{
<presence from='juliet@example.com/balcony'
to='benvolio@example.net'>
<show>away</show>
</presence>
}}}
And here's the same information in a JSON rendition:
{{{
["presence",{"from":"juliet@example.com/balcony","to":"benvolio@example.net"},{"show":"away"}]
}}}
Here's the equivalent PSYC packet in verbose form
(since it is a multicast, the single recipients do not
need to be mentioned):
{{{
:_context psyc://example.com/~juliet
=_degree_availability 4
_notice_presence
|
}}}
And the same in compact form:
{{{
:c psyc://example.com/~juliet
=da 4
np
|
}}}
=== An average chat message ===
{{{
<message
from='juliet@example.com/balcony'
id='ktx72v49'
to='romeo@example.net'
type='chat'
xml:lang='en'>
<body>Art thou not Romeo, and a Montague?</body>
</message>
}}}
=== A new status updated activity ===
Example taken from http://onesocialweb.org/spec/1.0/osw-activities.html
You could call this XML namespace hell:
{{{
<iq type='set'
from='hamlet@denmark.lit/snsclient'
to='hamlet@denmark.lit'
id='osw1'>
<pubsub xmlns='http://jabber.org/protocol/pubsub'>
<publish node='urn:xmpp:microblog:0'>
<item>
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:activity="http://activitystrea.ms/spec/1.0/"
xmlns:osw="http://onesocialweb.org/spec/1.0/">
<title>to be or not to be ?</title>
<activity:verb>http://activitystrea.ms/schema/1.0/post</activity:verb>
<activity:object>
<activity:object-type>http://onesocialweb.org/spec/1.0/object/status</activity:object-type>
<content type="text/plain">to be or not to be ?</content>
</activity:object>
<osw:acl-rule>
<osw:acl-action permission="http://onesocialweb.org/spec/1.0/acl/permission/grant">
http://onesocialweb.org/spec/1.0/acl/action/view
</osw:acl-action>
<osw:acl-subject type="http://onesocialweb.org/spec/1.0/acl/subject/everyone"/>
</osw:acl-rule>
</entry>
</item>
</publish>
</pubsub>
</iq>
}}}
http://activitystrea.ms/head/json-activity.html proposes a JSON encoding
of this. We'll have to add a routing header to it.
{{{
["activity",{"from":"hamlet@denmark.lit/snsclient"},{"verb":"post",
"title":"to be or not to be ?","object":{"type":"status",
"content":"to be or not to be ?","contentType":"text/plain"}]
}}}
http://about.psyc.eu/Activity suggests a PSYC mapping for activity
streams. Should a "status post" be considered equivalent to a presence
description announcement or just a message in the "microblogging" channel?
We'll use the latter here:
{{{
:_context psyc://denmark.lit/~hamlet#_follow
:_subject to be or not to be ?
:_type_content text/plain
_message
to be or not to be ?
|
}}}
=== A message with JSON-unfriendly characters ===
{{{
<message
from='romeo@example.net/orchard'
id='sl3nx51f'
to='juliet@example.com/balcony'
type='chat'
xml:lang='en'>
<body>"Neither, fair saint, if either thee dislike.", he said.
And
the
rest
is
history.</body>
</message>
}}}
=== A message with XML-unfriendly characters ===
{{{
<message
from='juliet@example.com/balcony'
id='z94nb37h'
to='romeo@example.net'
type='chat'
xml:lang='en'>
<body>Wherefore art thou, Romeo?</body>
<body xml:lang='cs'>
Pro&#x010D;e&#x017D; jsi ty, Romeo?
</body>
</message>
}}}
=== A message with PSYC-unfriendly strings ===
{{{
<message
from='juliet@example.com/balcony'
id='c8xg3nf8'
to='romeo@example.net'
type='chat'
xml:lang='en'>
<subject>I implore you with a pointless
newline in a header variable</subject>
<body>Wherefore art thou, Romeo?
|
And for practicing purposes we added a PSYC packet delimiter.</body>
</message>
}}}
=== A packet containing a JPEG photograph ===
... TBD ...
=== A random data structure ===
In this test we'll not consider XMPP at all and simply compare the
efficiency of the three syntaxes at serializing a typical user data base
storage information. We'll again start with XML:
{{{
<UserProfile>
<Name>Silvio Berlusconi</Name>
<JobTitle>Premier</JobTitle>
<Country>I</Country>
<Address>
<Street>Via del Colosseo, 1</Street>
<PostalCode>00100</PostalCode>
<City>Roma</City>
</Address>
<Page>http://example.org</Page>
</UserProfile>
}}}
In JSON this would look like this:
{{{
["UserProfile",{"Name":"Silvio Berlusconi","JobTitle":"Premier","Country":"I","Address":
{"Street":"Via del Colosseo, 1","PostalCode":"00100","City":"Roma"},"Page":"http://example.org"}]
}}}
Here's a way to model this in PSYC:
{{{
:_name Silvio Berlusconi
:_title_job Premier
:_country I
:_address_street Via del Colosseo, 1
:_address_code_postal 00100
:_address_city Roma
:_page http://example.org
_profile_user
|
}}}
== Conclusions ==
... TBD ...
== Criticism ==
Are we comparing apples and oranges? Yes and no, depends on what you
need. XML is a syntax best suited for complex structured data in
well-defined formats - especially good for text mark-up. JSON is a syntax
intended to hold arbitrarily structured data suitable for immediate
inclusion in javascript source codes. The PSYC syntax is an evolved
derivate of RFC 822, the syntax used by HTTP and E-Mail, and is therefore
limited in the kind and depth of data structures that can be represented
with it, but in exchange it is highly performant at doing just that.
So it is up to you to find out which of the three formats fulfils your
requirements the best. We use PSYC for the majority of messaging where
JSON and XMPP aren't efficient and opaque enough, but we employ XML and
JSON as payloads within PSYC for data that doesn't fit the PSYC model.
For some reason all three formats are being used for messaging, although
only PSYC was actually designed for that purpose.
== Caveats ==
In every case we'll compare performance of parsing and re-rendering
these messages, but consider also that the applicative processing
of an XML DOM tree is more complicated than just accessing
certain elements in a JSON data structure or PSYC variable
mapping.
For a speed check in real world conditions which also consider the
complexity of processing incoming messages we should compare
the performance of a chat client using the two protocols,
for instance by using libpurple with XMPP and PSYC accounts.
To this purpose we first need to integrate libpsyc into libpurple.
== Futures ==
After a month of development libpsyc is already performing pretty
well, but we presume various optimizations, like rewriting parts
in assembler, are possible.

View file

@ -0,0 +1,3 @@
["activity",{"from":"hamlet@denmark.lit/snsclient"},{"verb":"post",
"title":"to be or not to be ?","object":{"type":"status",
"content":"to be or not to be ?","contentType":"text/plain"}}]

View file

@ -0,0 +1,7 @@
:_context psyc://denmark.lit/~hamlet#_follow
:_subject to be or not to be ?
:_type_content text/plain
_message
to be or not to be ?
|

View file

@ -0,0 +1,27 @@
<iq type='set'
from='hamlet@denmark.lit/snsclient'
to='hamlet@denmark.lit'
id='osw1'>
<pubsub xmlns='http://jabber.org/protocol/pubsub'>
<publish node='urn:xmpp:microblog:0'>
<item>
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:activity="http://activitystrea.ms/spec/1.0/"
xmlns:osw="http://onesocialweb.org/spec/1.0/">
<title>to be or not to be ?</title>
<activity:verb>http://activitystrea.ms/schema/1.0/post</activity:verb>
<activity:object>
<activity:object-type>http://onesocialweb.org/spec/1.0/object/status</activity:object-type>
<content type="text/plain">to be or not to be ?</content>
</activity:object>
<osw:acl-rule>
<osw:acl-action permission="http://onesocialweb.org/spec/1.0/acl/permission/grant">
http://onesocialweb.org/spec/1.0/acl/action/view
</osw:acl-action>
<osw:acl-subject type="http://onesocialweb.org/spec/1.0/acl/subject/everyone"/>
</osw:acl-rule>
</entry>
</item>
</publish>
</pubsub>
</iq>

View file

@ -0,0 +1,6 @@
:_source psyc://example.com/~juliet
:_target psyc://example.net/~romeo
_message_private
Art thou not Romeo, and a Montague?
|

View file

@ -0,0 +1,8 @@
<message
from='juliet@example.com/balcony'
id='ktx72v49'
to='romeo@example.net'
type='chat'
xml:lang='en'>
<body>Art thou not Romeo, and a Montague?</body>
</message>

View file

@ -0,0 +1,13 @@
<message
from='romeo@example.net/orchard'
id='sl3nx51f'
to='juliet@example.com/balcony'
type='chat'
xml:lang='en'>
<body>"Neither, fair saint, if either thee dislike.", he said.
And
the
rest
is
history.</body>
</message>

View file

@ -0,0 +1 @@
["presence",{"from":"juliet@example.com/balcony","to":"benvolio@example.net"},{"show":"away"}]

View file

@ -0,0 +1,5 @@
:_context psyc://example.com/~juliet
=_degree_availability 4
_notice_presence
|

View file

@ -0,0 +1,4 @@
<presence from='juliet@example.com/balcony'
to='benvolio@example.net'>
<show>away</show>
</presence>

View file

@ -0,0 +1,12 @@
<message
from='juliet@example.com/balcony'
id='c8xg3nf8'
to='romeo@example.net'
type='chat'
xml:lang='en'>
<subject>I implore you with a pointless
newline in a header variable</subject>
<body>Wherefore art thou, Romeo?
|
And for practicing purposes we added a PSYC packet delimiter.</body>
</message>

View file

@ -0,0 +1,2 @@
["UserProfile",{"Name":"Silvio Berlusconi","JobTitle":"Premier","Country":"I","Address":
{"Street":"Via del Colosseo, 1","PostalCode":"00100","City":"Roma"},"Page":"http://example.org"}]

View file

@ -0,0 +1,10 @@
:_name Silvio Berlusconi
:_title_job Premier
:_country I
:_address_street Via del Colosseo, 1
:_address_code_postal 00100
:_address_city Roma
:_page http://example.org
_profile_user
|

View file

@ -0,0 +1,11 @@
<UserProfile>
<Name>Silvio Berlusconi</Name>
<JobTitle>Premier</JobTitle>
<Country>I</Country>
<Address>
<Street>Via del Colosseo, 1</Street>
<PostalCode>00100</PostalCode>
<City>Roma</City>
</Address>
<Page>http://example.org</Page>
</UserProfile>

View file

@ -0,0 +1,11 @@
<message
from='juliet@example.com/balcony'
id='z94nb37h'
to='romeo@example.net'
type='chat'
xml:lang='en'>
<body>Wherefore art thou, Romeo?</body>
<body xml:lang='cs'>
Pro&#x010D;e&#x017D; jsi ty, Romeo?
</body>
</message>

View file

@ -1,26 +0,0 @@
#+TITLE: Benchmark results
#+OPTIONS: ^:{} toc:nil
* Results
Parsing time of 1 000 000 packets in milliseconds:
| input | PSYC | | JSON | | | XML | |
| parser | strlen | libpsyc | json-c | json-glib | libxml sax | libxml | rapidxml |
|-----------+--------+---------+--------+-----------+------------+--------+----------|
| presence | 30 | 246 | 2463 | 10197 | 4997 | 7557 | 1719 |
| chat msg | 41 | 320 | | | 5997 | 9777 | 1893 |
| activity | 42 | 366 | 4666 | 16846 | 13357 | 28858 | 4419 |
| user prof | 55 | 608 | 4715 | 17468 | 7350 | 12377 | 2477 |
|-----------+--------+---------+--------+-----------+------------+--------+----------|
| / | < | > | < | > | < | | > |
* Commands used
: ./testPsycSpeed -sc 1000000 -f $file
: ./testJson -snc 1000000 -f $file
: ./testJsonGlib -snc 1000000 -f $file
: ./testStrlen -sc 1000000 -f $file
: ./rapidxml 1000000 $file
: ./libxml 1000000 $file
: ./libxml-sax 1000000 $file