• first had to prepare a couple of talks
  • Georgi Kobilarov kindly asked me to give a presentation at the Web of Data Meetup Berlin
  • Then i was invited to give a talk at the IFRA Newsroom Summit in London (right before Arthor Sulzberger Jr.’s speaking slot :-)
• then went for a two week holiday (Sardegna, – very nice),
• had three weeks of
  • conferences  (Scoopcamp, Scoopcamp Hackathon),
  • fairs (IFRA Expo),
  • meetings (MINDS International) and
  • workshops (BDZV – Tablets für Blattmacher) and
• did the preparation for next years budgeting.
• did some more talks
  • an invited  dinner talk at the Cassini Convention (a years end  customer event )
  • and a talk at the Innovation-Forum (a course in the brand news Masters study IT-Management and Consulting) at the CS department at the University of Hamburg
• and participated in the 1st Speedlab journalism (as an expert on “The future of news” :-)

All in addition to running several projects: mostly focused on digital publishing, but also on (Geo-)Metadata and other stuff.

In my spare time i continued renovating the house and started setting up my new desktop computer (The first desktop i bought in more than ten years). Actually it is no longer a desktop since i wall mounted it. For those interested: it’s quite a beast (27” iMac, Core i7, 16GB RAM, SSD + 1 TB) that should povide the processing power for the experiments i want to run this year.

So all in all it was (and still is) a quite busy time.

I hope in the meantime you enjoyed my Google shared item feed, my delicious bookmark feed , and my twitter feed

But now i’m back and intend to spent more time on blogging (and building  stuff) again.

BTW.: I switch to doing an svn export of the WP tag from the automattic repository followed by an internal rsync -avz. This reduces the upgrade time to a couple of seconds. Due to the layout of WP (mixing original files and  user adaptions in the same directory this also means that there is the chance of stale files at the rsync target directory, perhaps introducing potential security risks.

Wrt. 2.6  I’m especially interested in the revisions and the PressThis functionality as well as the image captions. Right now i’m sharing interesting blog posts in my  google shared item feed. I’m definitely not going to do a press this post for every blog post i’m sharing but try to find a middle ground. In preparation for this i added a new category “Fresh from the press“.

Initially i’m going to make the PressThis posts visible at the homepage but will eventually hide them from it, similar to the DailyDeli posts. If don’t know about the DailyDeli posts. These are post that are automatically generated by delicious and cover all my daily bokmarking at delicious. BTW. my delicious account can be found http://del.icio.us/gkamp.

In other news my apologies for not posting more often recently but i had to work on renovating the house and some other stuff i’m working on. Hopefully this will change soon.

But expect a number of articles tagged goingplaces this week.

Warning: This may be boring stuff for some of you and you might ask yourself why is this guy trying so hard to find kinda formal definitions. Actually doing this it is fascinating stuff for me (couldn’t negligect help my former AI, ontology building life).

In addition i think it is absolutely essential to have an idea of how and why you are adding geocodes to your content before you are going to do so. Especially when you are a professionnal news organization and reseller. So here we go:

Geocoded News Stories

The first thing to do is to somehow define what a geocoded news story is about. Please note that all following definitions do not attempt to fulfill all the criteria a mathematician has in mind when he hears the term definition but rather to create a comon understanding of what is meant, e.g. more like a glossary entry / dictionary entry.

A geocoded news story

is a news stories that has at least one location attached as accompanying meta-data. Locations include locations of the news story as well as locations in the news story.

In case the story is a complex news story composed of a number of parts (e.g. multiple texts and / or multiple images, a multimedia news story consisting of text, image, video, audi, etc). the locations of the story as a whole is the (multi-)set of the parts of the story.

The most important things to note is that a news story can have multiple locations and tehes locations can have different roles wrt. the story.

Locations of news stories, Locations in news stories

First a kinda formal distinction between locations of news stories and locations in news stories:

A location of a (news) story

is a location assigned to the news story as a whole. This location is not necessarily mentioned in the story itself.

There are different types of locations of a news story, e.g. scope, locus and place of production (see below). Whereas scope and locus are generally geographic names (in our case mostly administrative divisions like states, counties, cities but also other geographic features that have a geographic name), a place of production usually is more specific, e.g. an address.

A location (with)in a (news) story

is a location directly or indirectly mentioned in the news story itself.

Most of the locations in news stories do not fullfil the criteria of a geographic name (see below). They are most likely rather of thekind of business and private addresses, street segments, blocks, or points of interest. Not surprisingly, locations within news stories are typically more specific than locations of news stories (in the sense of having a smaller geographic extent).

You might wonder why i added the indirectly mentioned in the definition. This is in order to take care of the fact that typically not the location itself, but a business, a governmental institution name like a court etc. is mentioned in the story. The actual locations, i.e. the addresses of the relevant branches / offices of a business are not part of the story and have often have to be inferred from the text of the story.

Actually, if you are writing the story you definitely know what locations you mean. Unfortunately, today,this kind of meta-data is almost never attached to the story at the source. I’m currently working hard at work to

• raise the awareness within our and other news organization that this meta-data is important information and actually a USP
• look for ways to include this kind of information into our legacy systems without breaking them
• building tools for enabling the writers to easily add the locations to their stories (e.g looking up addresses in interna and external directories etc.

Geographic Names

As you can see from the above remarks the term geographic name is of special importance in this context, hence it should be defined. we use it according to the definition of the U.S. Board on Geographic Names (from Principles, Policies, and Procedures for Domestic Geographic Names)(emphasis by me):

A geographic name

is a name applied to a geographic feature. It is the proper name, specific term, or expression by which a particular geographic entity is, or was, known. A geographic entity is any relatively permanent part of the natural or manmade landscape or seascape that has recognizable identity within a particular cultural context. A geographic name, then, may refer to any place, feature, or area on the Earth’s surface, or to a related group of similar places, features, or areas.

Scopes

As said above, locations of news stories can be of different types. There are at least the following types: scope, locus and place of production. There are surely additional types of places of news stories i don’t know about. The following are the current definitions of Scope, Locus and Place of production we use. First the definition of scopes (Warning: this is going to be a long section)

A scope of a story

is a geographic name that is part of an (official administrative divisions) hierarchical partition of a defined geographic extent, representing a largest area wrt. the above hierarchy where this story is deemed relevant by an editor

Rem.:

A variant of scopes are legal scopes. These do not describe the area of relevancy but the geographic extent in which it legally allowed to use this content. This is especially often the case with images and other non-textual content.

The basic intention of scopes is to be able to describe a geographic extent as unambigously as possible without actually describing the geometry, while in parallel using a terminology that is known to ordinary people.

Using scopes instead of the actual geometries frees on one hand our customers from having the need to have their own GIS infrastructure in place. Most often the communicated identifiers are sufficient for enabling them to solve their needs. If they need real geographic inferences they are still able to buy the underlying data itself and /or use a webservice to retrieve the geometries.

On the other hand using identifiers and not the geometries is often the only way to enable to communicate geographic extents to our customers. At least in germany it is not possible or very expensive to buy the redistribution rights to the underlying geometries. It alo saves a lot of bandwidth.

Please note that a news story can have multiple scopes and that not all scopes have to be in the same hierarchy. It is only required that every hierarchy in itself is a hierachical partition of a clearly defined geographical extent.

It is absolutely perfect to have one scope being contained in a hierarchy e.g. denoting the administrative divisions of germany as defined in the so called “Amtlicher Gemeinde Schlüssel (AGS)” (Bundesländer ~ states, Regierungsbezirk (no equivalent in the US), Kreis ~ county, Stadt/Gemeinde ~city/town/village) and another scope belonign to a second hierarchy denoting the adminstrative subdivisions of a certain city, e.g the boroughs and districts of Hamburg, the neigbourhoods of hamburg as defined by some company or community, the zipcodes of germany etc.

I also think that being able to add metadata describing a geographic extent to which content is deemed relevant would benefit all kinds of content, ranging from tweets, (e.g. please notify only my friends in the City of San Franciso that i’m coming to town, because i’m only there for 2 hours, and other friends in California wouldn’t make it in time) to blog posts since they are basically news stories to wikipedia entries.

Another way of looking at scopes is as hints of what to expose at what zoom level on a map. For doing so you don’t need complex calculations. Adding some information/ access to the bounding box of the scope in order to be able to do so,

Hierarchical partitions

Since my early experiences i don’t believe in grand unified theories / ontologies, that try to the model of a domain. I rather believe in sets of small, very domain specific ontologies. The notion of a hierarchical partition for a certain extent does originate from this belief. It encapsulates the partonomy relationships for localities of a coherent set of types.

From an engineering point of view the notion of a hierarchical partition also allows us to loosely couple the different hierarchies.

So what is a hierarchical partition of a defined geographic extent? And why do we care? To answer the second question first:

• We have somehow to explain what we are doing to our customers (and ourselves)
• If we happen to come up with a definition that has a nice set of properties we might be able to use algorithms that take advantage of these properties. The following shows that our understanding / definition of what a hierarchical partition is evolved over time.

I first interpreted hierachical partition in the pure mathematical sense, i.e.:

• for any given point in the plane within the defined geographic extent there is exactly one corresponding scope on each level of the hierarchy
• for any given scope there is exactly one predecessor wrt. this hierarchy.

But looking at the administrative division of germany i recognized that this is actually not the case. and the first criterion has to be relaxed. This stems from the following facts.

• There are counties denoting cities (so called Stadtkreise) that are not represented in the “city/town/” level of the AGS hierarchy. e-g there “holes” at this level. While this fact may be worked around by adding these counties into the city level of the hierarchy.
• Some states do not have so called “Regierungsbezirke”, they eliminated this level at some time in the past. Hence there are also holes at this layer.

The following changes of the rules would take care of these facts:

• for any given point in the plane within the defined geographic extent there is at least one corresponding scope in some level of the hierarchy
• for any given point in the plane within the defined geographic extent there is at most one corresponding scope on every level of the hierarchy
• for any given scope there is exactly one predecessor wrt. this hierarchy.

So i thought that this division was sufficient to cover also the adminsitrative subdivisision of other countries. But when validating this defintion against the administrative divisions of the United States i learned that New York City is an aggregate of 5 counties of the state of New York , each county being coterminous with a borough of New York City. Taking care of that and hopefully preparing ourselves of other “strange” cases we end up with the following definition of a hierarchical partition:

A hierarchical partition p of scopes of a geographic extent e

is a directed acyclic graph (DAG) with the following properties:

1. There is a single source s_top (the top level scope) with a geographic extent being coterminous with the geographic extent (using coterminous as having matching boundaries interpretation
2. every scope has a property denoting its level in the hierarchy with the top level scope having the level 1
3. for any given point p in e there is at least one corresponding scope s(point) at some level in the DAG
4. for every scope that has more than one successor the geographic extent of set of successors is coterminous with the geographic extent of this scope
5. for every scope that has more than one predecessor the geographic extent of set of predecessors is coterminous with the geographic extent of this scope

Rem:

• This definition is definitely not perfect in it’s formulation but some of you might help me with improving it. It might also be better to start with a poset based definition and switch to a graph based definitin when introducing additional relations, e.g. topological relations describing adjointness etc.
• It might be helpful to distinguish between hierarchical partitions and leveled hierarchical partitions with the difference between two two is the fact if the scopes are assigned levels, or not.
• Why is it important to have the level information you might ask? It is necessary in order to transcribe the semantics
• In order to not lose the two stricter definitions the first one is defined as a strict partition hierarchy, whereas the second is a partition hierarchy.
• I haven’t found the time to look deeper into this, but it looks like that it is likely that a (leveled) hierarchical partition already has been assigned a name somewhere in mathematics.  IIf someone out there happens to know where to look (computational geometry?) would love to know about it.
• It also seems to be the case that if you add a an additional level with a single node s_bottom that is the successor of every leaf node in the hierachical partition, you get a lattice.  Maybe some of the lattice properties and knowledge /algorithms for lattices might prove helpful

After this very extensive coverage of scopes i just briefly introduce the current definitions of loci and places of production. This is mostly the case because right now there are only some ideas how a definition of these should look like.

A locus of a story

• is a geographic name contained in a set of geonames of a defined geographic extent,
• representing the / a smallest area wrt. the above hierarchy where the events of this story are happening / have happened / are going to happen

Rem.:
Initially a locus was also defined a being part of a hierarchical partition. This gains the advantage to being able to unambigously describe the locus withtin that hierachy (at least at each level). But while this is a property that is important for scopes, in fact that is the main purpose of scopes, being able to use names that are typically used e.g natural features like mountain ranges etc. is more important than being unambigous.

A place of production of a story

• is a location where the news story (or parts of it) were produced (e.g. written by the author, edited by the editor, …)
• describing as exact as possible the geographic position of the production (e.g. using geographic coordinates, addresses, …)

Locations within news stories

A location (with)in a (news) story is a location directly or indirectly mentioned in the news story itself. These locations are typically not geographic names but rather addresses, street segments, blocks, or POIs. Not surprisingly, locations within news stories are typically more specific (in the sense of having a smaller geographic extent) than locations of news stories.

What’s next?

In the next post i’m sketching the current status and am finally giving you some examples.

But then the problems started. After the upgrade i had a look at the homepage and there were no stories to see. I immediately suspected the plug-ins. After disabling all plug-ins the stories were there again, hence the suspicion was right.

I only installed and activated a few plug-ins:

Ultimate Tag Warrior

for tagging the posts,

Spam Karma

for catching spam,

TinyMCE advanced

for enhanced editing

Ted Fox’s plugin for

changing the thumbnail size

the category visibility plugin

for excluding the automated posts from del.icio.us from the frontpage and the feeds.

Since WP2.5 should include the functionality of of UTW and the changing the size i could do without them. But first i had to figure out how to convert the UTW Tags. The respective Import > Ultimate Tag Warrior command did work seamlessly for importing the tags. Since i haven’t uploaded any images, i couldn’t check if the included WP2.5 functionality is equivalent to the plug-in functionality. We’ll see.

Unfortunately there is no officially approved Spam Karma 2 for WP2.5. But i hope that the version i’m using right now is continuing to work as effectively as it did, this blog could not live without it. And no Akismet is not considered an option, i like to have this blog self-hosted and not dependent on third-party services for mission critical parts. But i think it is fairly save to bet that there are no changes in WP2.5 that affect Spamm Karma 2.

That leaves the category visibility plugin and TinyMCE Advanced. I used the auto-upgrade option for TinyMCE Advanced (first time i’ve seen this). This lead to a state where i was simply not able save a post. Everytime i tried to save the post, i got the “Do you really want to leave this page, you’ll lose all your changes, ..” pop up alert. Since i’m not dependent on the added functionality of TinyMCE Advanced, i keep it deactivated for now and have a look into it later.

The category visibility plugin does not work with WP2.5, as the category and tag system has been changed . It was also the most likely suspect for the homepage not displaying any post. I then looked for an upgraded version, and had to find out that the orginal author abandoned development, and the patch for WP2.3 he linked from his site went out of business too. Hence i had to leave it with all categories enabled on both the frontpage and the feeds for the time being. I’m not sure if this bothered you or just reminded you that there is also a DailyDeli category :-). I finally found a version of that plugin that seems to work with WP2.5. Hence the dailyDeli posts should be removed from the FrontPage and the feeds. If would like to have the DailyDeli posts included in the feeds please tell me.

All in all WP upgrades are IMHO too much hassle. This starts with the fact that i don’t understand why plugins and themes are located below wp-content and the whole structure of WP is not very upgrade friendly, Then there is no support for easily detecting which versions of the plugins are compatible with the new version etc. Automattic should definitely have a look at Typo3 how they handle this. For example, this is the layout of the typo3 root directory:

-rw-r--r-- 1 p34755284 ftpusers    46 Sep  7  1999 clear.gif
drwxr-xr-x 8 p34755284 ftpusers    97 Nov 20  2005 fileadmin
-rw-r--r-- 1 p34755284 ftpusers   764 Apr 18  2003 index.html
lrwxrwxrwx 1 p34755284 ftpusers    18 Jan  7 01:36 index.php -> tslib/index_ts.php
lrwxrwxrwx 1 p34755284 ftpusers    12 Jan  7 01:36 media -> tslib/media/
lrwxrwxrwx 1 p34755284 ftpusers    17 Jan  7 01:36 showpic.php -> tslib/showpic.php
lrwxrwxrwx 1 p34755284 ftpusers    16 Jan  7 01:36 t3lib -> typo3_src/t3lib/
lrwxrwxrwx 1 p34755284 ftpusers    16 Jan  7 01:36 tslib -> typo3_src/tslib/
lrwxrwxrwx 1 p34755284 ftpusers    16 Jan  7 01:36 typo3 -> typo3_src/typo3/
lrwxrwxrwx 1 p34755284 ftpusers    32 Jan 25 08:20 typo3_src -> ../../code/typo3/typo3_src-3.8.1
drwxr-xr-x 3 p34755284 ftpusers  4096 Aug 24  2007 typo3conf
drwxr-xr-x 6 p34755284 ftpusers 16384 Mar 12 14:34 typo3temp
drwxr-xr-x 8 p34755284 ftpusers  4096 Nov 20  2005 uploads

What at first sight looks like an awful lot of unnecessary symbolic links, is actually a clever scheme that allows to upgrade to a new version of Typo3 with pointing a single link: typo3_src to a different directory while keeping all the configuration data and extensions in cleanly separated in typo3conf and all the uploaded content also in separated containers, e.g. fileadmin

Update:

Looks a lot that the WordPress Automatic Update plugin would have saved me a lot of trouble. I just noticed that i forgot to include the Google Analytics tracking code after updating the themes for WP2.5

Is there a plugin for inserting the GA tracking code?

A couple of weeks ago i made a small change to the way full size images are displayed. I’m now using FancyZoom, an excellent opensource way of javascript-enabling / web2.0-ifying this process. Since it leaves the html code completely untouched it should work seamlessly with feed readers, and degrade gracefully on old browsers. If you experience any problems, please tell me.

Tapping into my shared items

Some of you might have noticed that i make extensive use of the “Share item” feature of Google reader. I think of it as a service to the readers of this blog: An intelligent filter to the one-hundred smething feeds i’m reading. To be exact (according to my Google reader trends page):

From your 182 subscriptions, over the last 30 days you read 5,933 items, starred 0 items, shared 499 items, and emailed 0 items.

With my last post i decided that it makes sense to tap into this ressource while writing posts. In some sense i already did this in some way for quite some time. I used the search feature of google reader (filtered on my shared items) to find the respective blog posts.

I now decided to include a select list of the entries returned by these searches as a “See also service”. Please tell me if you consider this avaluable add on.right now i’m not sure if it makes sense to include the abstracts of the related posts. Please tell me.

The process itself went smoother than anticipated. I ordered the cross-grade on Jan, 3rd. and was told that it would take approx. 5- 7 workdays.

Just to be sure first i did backup my MySQL-databases. There were some minor problems with the phpmyadmin supplied by 1&1 not being able to backup the database of this blog. Presumably because it was too big (or taking too long) due to all the Spam-Entries in the database. I had to resort ssh and the commandline (actually a much nicer process ;-) I then did a backup of the dumped databases and the webspace to my local machines via rsync.

Expecting something like a mail announcing that the maschine will be switched in the next 24 hours, i waited. On Jan, 7th just by curiosity i logged into the machine and did a “less /proc/cpuinfo” and found that the machine was already upgraded.

I tested the different domains and apps and it looks that all are running smoothly. So if you find any glitches, contact me.

Right now I’m not satisfied with the look of the page. It neither a natural translation of the the veryplaintext theme i use, nor a good implementation of the iPhone human interface guideline. But it’s a starting point.

If anybody knows a better iPhone theme, i’d like to hear.

BTW.: Does anybody know it there is a way to autmatically t focus the page in a way that the addressbar is hidden? I find it a bit annoying (especially in landscape mode)

So i finally settled on doing it. Fortunately there is already a plugin for that purpose called catvis. Unfortunately some people mention that it might cause problems with WP2.2+. And indeed it does. After unchecking the box for showing the DailyDeli on the front page everything seemed fine at first, there where problems while displaying the single post pages.

The error message clearly showed that there where unbalanced parantheses in the generated SQL-Code. Easy to fix i thought. Hence i changed:

elseif (is_single()) // add permalink check (eff beta 10, thanks to ruralmoon)
$ret .= " AND (($cat_visibility.archives=1 AND $lc";

into

elseif (is_single()) // add permalink check (eff beta 10, thanks to ruralmoon)
$ret .= "AND (($cat_visibility.archives='$checkval' AND $lc)";

If you still find any erros, please report them and i may have a closer look at the plugin code. But since i’m not a PHP guide (and don’t want to become one, i try to avoid this.
matching the code for some other cases. But then wordpress complained about the DB-table not being there.

WordPress database error: [Unknown table 'wp_catt_visibility' in where clause]
SELECT wp_posts.* FROM wp_posts WHERE 1=1 AND ID = 278 AND (post_type = 'post') AND ((wp_catt_visibility.archives='1' AND wp_catt_visibility.cv_user_level<='10') OR post_type='page' OR wp_catt_visibility.catt_ID IS NULL) ORDER BY post_date DESC

and

WordPress database error: [You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near 'ORDER BY post_date ASC LIMIT 1' at line 1]
SELECT ID, post_title FROM wp_posts WHERE post_date > '' AND post_type = 'post' AND post_status = 'publish' AND ID != ORDER BY post_date ASC LIMIT 1

Whereas the DB-Table wp_catt_visibility is definitely defined in DB, the second error clearly exposes an error in the SQL-Code. Since i figured that i never would need the ability to hide a single post i just uncommented it ending up with:

elseif (is_single()) // add permalink check (eff beta 10, thanks to ruralmoon)
// GK: do nothing because of WP complaining about the statement below
$ret .= "";
//GK: fixed unbalanced parantheses
//$ret .= "AND (($cat_visibility.archives='$checkval' AND $lc)";

In related news, i also finally enabled permalinks (after checking that the old unrewritten links still work

If your still experience any problems with this site problems please report them to me.