RTML Problems? Missing Expressions May be the Cause

Today’s Y! Store Blog post comes from Adam Davis, a technical account manager with our customer care team here at Yahoo! Small Business. Below, he shares some of his RTML knowledge to help Yahoo! Store merchants troubleshoot a common issue encountered when building custom RTML templates. Merchants are reminded that Yahoo! Small Business does not offer support for custom templates or RTML customization. The post is recommended for merchants with some understanding of and experience with RTML.

A popular method of building templates in RTML involves using a text field filled with HTML, then placing words inside these HTML tags that are unique enough to be used as an identifier or "token."

Below is an example of an RTML field that contains HTML, which I’ve taken from my own store. In my store variables, I have a med-text field named "site-html." It contains the following HTML:

<div id="main">
<div id="top-nav"> @@TOP-NAVIGATION@@ </div>
<div id="body">
<p>@@CAPTION@@</p>
</div>
<div id="left-nav"> @@LEFT-NAVIGATION@@ </div>
</div>

In my templates, I call my variable site-html. I use TOKENS to iterate over each piece of the HTML, then execute something where there is a match for my SWITCH keys.

<RTML>
     WITH= variable site-html
             value @site-html
         FOR-EACH var html
                 sequence TOKENS site-html
             SWITCH html
Key          "@@TOP-NAVIGATION@@"
expression       CALL :NAVIGATION
                 "top"
Key          "@@LEFT-NAVIGATION@@"
expression       CALL :NAVIGATION
                 "left"
Key          html
expression       TEXT html
</RTML>

There is a major pitfall when working with this type of template because SWITCH is evaluated from top to bottom, interpreted as alternating key-expression pairs. When a switch expression is evaluated, its value is matched against the keys, the corresponding expression is evaluated, and its value is returned.

When using TOKENS to go over a large block of data, you need to be very careful with the construction of your SWITCH. The most common cause of trouble with a SWITCH is a missing expression.

Taking the above example and adding a new key with no expression will continue to work in Store Editor. However, it will also cause a large number of errors that may result in being unable to publish the site.

<RTML>
     WITH= variable site-html
             value @site-html
         FOR-EACH var html
                 sequence TOKENS site-html
             SWITCH html
Key          "@@TOP-NAVIGATION@@"
Expression       CALL :NAVIGATION
                 "top"
Key          "@@LEFT-NAVIGATION@@"
expression       CALL :NAVIGATION
                 "left"
Key          "@@CAPTION@@"
expression       html
Key          TEXT html
</RTML>

This RTML will outwardly work the same as the first example, but there will be an extra overhead for the template due to the missing expression.

It will evaluate something like this:

no case for:"<div"
no case for:"id=\"main\">"
no case for:"<div"
no case for:"id=\"top-nav\">"
@@TOP-NAV@@ <– will evaluate successfully and call the navigation template.
no case for:"</div>"
no case for:"<div"
no case for:"id=\"body\">"
no case for:"<p>"
@@CAPTION@@ <– will do nothing because when this is called it is in an unactionable context ie: html = "@@CAPTION@@" which does nothing.
no case for:"</p>"
no case for:"</div>"
no case for:"<div"
no case for:"id=\"left-nav\">"
@@LEFT-NAVIGATION@@ <– will evaluate successfully and call the navigation template.
no case for:"</div>"
no case for:"</div>"

As you can see, there are 14 extra lines of overhead from a very simple piece of HTML.

Now you may ask, "Mr. Yahoo, I would like to build out my all my SWITCH Keys, but they do not yet have an expression to evaluate. What do I do?"

The answer is, use "nil,” as in the example below:

<RTML>
     WITH= variable site-html
             value @site-html
         FOR-EACH var html
                 sequence TOKENS site-html
             SWITCH html
Key          "@@TOP-NAVIGATION@@"
expression       CALL :NAVIGATION
                 "top"
Key          "@@LEFT-NAVIGATION@@"
expression       CALL :NAVIGATION
                 "left"
Key          "@@CAPTION@@"
expression       nil
Key          html
expression       TEXT html
</RTML>

With one more line of RTML, every SWITCH key now has an expression to evaluate, and no errors are returned.

The second situation that can cause a very difficult problem to troubleshoot, yet is easy to see, is a missing expression in the middle of the SWITCH causing garbled output.

<RTML>
     WITH= variable site-html
             value @site-html
         FOR-EACH var html
                 sequence TOKENS site-html
             SWITCH html
Key          "@@TOP-NAVIGATION@@"
expression       CALL :NAVIGATION
                 "top"
Key          "@@LEFT-NAVIGATION@@"
expression       "@@CAPTION@@"
Key          TEXT @caption
expression       html
Key          TEXT html
</RTML>

This will result in @@TOP-NAVIGATION@@ being evaluated correctly, however every space/tab/ (and in this case @@LEFT-NAVIGATION@@) in the HTML will be replaced with a copy of the caption.

In this example my caption is TEXT " This is my caption " and my top navigation is TEXT "*  top navigation bar  *".

–HTML output–
<div This is my caption id="main"> This is my caption
<div This is my caption id="top-nav">* top navigation bar * This is my caption </div>
This is my caption <div This is my caption id="body"
This is my caption <p>@@CAPTION@@</p>
This is my caption </div>
This is my caption <div This is my caption id="left-nav"> This is my caption </div> This is my caption
</div>
–end–

The fix for this issue? Like with the first example, make sure that all of your SWITCH keys have an expression.

<RTML>
     WITH= variable site-html
             value @site-html
         FOR-EACH var html
                 sequence TOKENS site-html
             SWITCH html
Key          "@@TOP-NAVIGATION@@"
expression       CALL :NAVIGATION
                 "top"
Key          "@@LEFT-NAVIGATION@@"
expression       CALL :NAVIGATION
                 "left"
Key          "@@CAPTION@@"
expression       TEXT @caption
Key          html
expression       TEXT html
</RTML>

I hope these examples assist you with troubleshooting some of your RTML customizations. If you’re not comfortable working with RTML, but wish to customize your store, check out our Yahoo! Merchant Solutions Developer Network for developers who specialize in working with Yahoo! stores.

Adam Davis
Yahoo! Small Business

Speed up the publish time of your store

The following post comes courtesy of Istvan Siposs, Yahoo! Store developer, RTML guru, and author of several invaluable guides for merchants such as the Yahoo! Store Tips & Tricks 2nd edition, and RTML 101: The Unofficial Guide to Yahoo! Store Templates. The post is recommended for merchants with some understanding and experience with RTML.

Recently I did a complete redesign in a store. As part of the redesign, I added a hierarchical, DHTML menu navigation bar to the site. In a typical setup, such a navigation bar works like this: use WITH-OBJECT :index to reference the home page, then cycle through the contents field, create a menu label for each page you find there, and for any such page, create a sub-menu if the page also has contents.

If you didn’t know until now, whenever you reference another object in RTML, such as using WITH-OBJECT or FOR-EACH-OBJECT, such a reference causes a disk lookup on the server. This is what we call an “expensive” operator; it is expensive in terms of processing power and time, so a navigation bar in general, and a hierarchical navigation bar in particular is an “expensive” template.

In the store I was working in, they had over 4,000 pages, and of course, each of those pages had the navigation bar on them. So not surprisingly, the time it took to publish this site went from a couple of minutes to 3-4 hours!

ONCE to the rescue

There are many ways to write good, efficient, and bad inefficient RTML code, but regardless of your coding style, there is one operator you should consider every time you write a piece of code that won’t change from page to page, and it is the ONCE operator.

ONCE takes one parameter, which can be either :page or :publish. What it means is that whatever you paste within ONCE will be evaluated only once per page or per publish. Now, per page is not that interesting (I think at least, I haven’t really found any use for that,) but per publish is great.

When you use ONCE :publish, the expressions pasted inside it are evaluated only once during publishing. So how can you speed up your navigation bar? The immediate idea is to simply paste your original navigation code inside a ONCE :publish. Ok, try it, and you’ll be quite surprised: your navigation bar will appear only on a single page, the first one that was generated during your publish. Not exactly what you want, but if you think about what ONCE does, it does make sense: you asked it to generate your navigation bar once per publish. To get the speed benefits of ONCE :publish and also end up with a navigation bar on every page, do the following small variation:


TEXT ONCE :publish

Now this will do the trick nicely! We still generate the navigation bar once per publish, but now the result is saved (GRABbed) and output on each page!

With this simple trick, my store’s publish time went back to about 5-10 minutes.

I use this trick all the time for pieces of the page that doesn’t have to change throughout the site. It works nicely for banners, footers, random testimonials, etc. You can use the above code snippet as a boiler plate, just put the code that needs to be evaluated once inside the GRAB operator and you’ll be all set. Be careful though, don’t do this to code that changes from page to page (for example, you cannot use this trick if your navigation bar changes based on which main category you are in.)

Editor still slow?

Ok, you’ve tried it, publish time is great, but your editor is slow. Chances are you have some inefficient RTML code in your templates, and if that’s the case, ONCE will not save you anything there. Why? Because when you are in the editor, you are essentially publishing each page every time you view a page! Not the entire store (of course), but viewing any page in the editor is a “mini publish” of that page. So if you take my DHTML menu example above, that menu needs to be re-generated in the editor every time you view a page. One way to speed up the editor is to turn off slow pieces of code while in the editor. For example, you can program your template so that it does not show a navigation bar while in the editor (in the editor, you can still navigate in other means, like a bottom of page text navigation, breadcrumbs, or clicking the “Contents” button.)

It is easy enough to turn off any piece of RTML code by using the WHEN operator. You can, for example, create a custom yes-no variable called “in-editor”, and then take your existing site navigation and stuff it inside a WHEN @in-editor operator. This will do the trick, simply set in-editor to Yes while you are working in the editor, and set it to No before publishing.

There are two problems with this approach though: one, you have to manually do this all the time you are working in the editor, and two, you or someone at your organization might accidentally publish the store with that setting set to Yes. Then, your live site will end up with no navigation. So what to do?

With a little trick, instead of using a custom variable, let the editor tell you whether your code is running in the editor or not! Create the following template:


is-editor

Once you have this template, use it instead of a custom variable to disable pieces of code in the editor like this:


WHEN NOT CALL :is-editor

... and disabled inside the editor

But how do I navigate?

There are many other ways to navigate your store besides using the store’s navigation bar. You can use the “Contents” page in the Advanced Editor, or use the footer text links (if you have those), or breadcrumbs. Or, if you know the ID of the page you want to edit, you can erase the last piece (after the last / character) of the address in the browser’s address box, and replace it with the page Id plus the .html extension. For example, if you want to go to the site map page, and the address in your editor’s address bar looks like this (more or less):

http://us-f4-edit.store.yahoo.com/RT/NEWEDIT.storeid/d49f35bb5d6b/C7FwkAAB

then replace the bolded part with ind.html and hit Enter like this:

http://us-f4-edit.store.yahoo.com/RT/NEWEDIT.storeid/d49f35bb5d/ind.html

Istvan Siposs,
Guest blogger for Yahoo! Small Business

Sordid Details, of a sort

The following post from Sheridan Rawlins, Yahoo! Small Business engineer, outlines two often misunderstood RTML operators, SORT, and INDEXED-SORT. The post is intended for Yahoo! Store Developers or merchants with an advanced understanding of RTML. Learn more about RTML.

Some time ago we released the SORT operator in response to the difficulty and tedium of writing a sorting template directly in the RTML language. I would like to shed some light on how it works, describe the differences in the two flavors of sort (SORT & INDEXED-SORT), and describe more advanced sorting, such as sorting on a secondary property when the first property is equivalent.

Both the SORT and INDEXED-SORT operators expect 2 variable names (var1 & var2), a sequence, and a body. The elements of sequence are examined 2 elements at a time by setting the two variables and evaluating the body to determine whether var1 is less than the var2. A copy of the sequence in sorted order is returned.

The body allows full customization of the sort order subject to some conventions which should be followed: return-style consistency, and comparison consistency.

Return-style consistency

We support two styles of body’s return value – one of the following styles should be chosen and stay consistent for a given instance of SORT or INDEXED-SORT:

1. strcmp-like: a number, which is
   1. -1 if var1 < var2
   2. 0 if var1 = var2
   3. +1 if var1 > var2
2. Binary Predicate: a symbol which is
   1. T if A < B
   2. NIL if A >= B

Comparison Consistency

The comparison must obey the following “StrictWeakOrdering” properties:

• Irreflexive: when (var1=A, var2=A), body must return 0 or NIL
• Antisemetric: when (var1=A, var2=B) returns -1 or T, then (var1=B, var2=A) must return +1 or NIL
• Transitive: when (var1=A, var2=B) returns -1 or T, and (var1=B, var2=C) returns -1 or T, then (var1=A, var2=C) must return -1 or T as well.

When to use SORT vs. INDEXED-SORT

SORT is useful when sequence directly describes the elements to be sorted.

INDEXED-SORT is useful when “expensive” lookups must be performed against each element in sequence to determine the index to sort on. The best example of this is when you have a sequence of ids (i.e. @contents) and you wish to sort on the @name property of each element.

INDEXED-SORT’s extra parameter: getindex

INDEXED-SORT first iterates across sequence setting var1 for each element and evaluating getindex. Then, it evaluates body much like sort but sets var1, and var2 to these indices. A copy of the sorted sequence is returned (indices are discarded).

I hope this helps describe the basic principles of the sort and comparison operators. I’ll provide examples and other sorting tidbits in a companion post.

Sheridan Rawlins
Yahoo! Small Business

Edit 01/11/07

In order to reverse the sort order you need to tweak the body of the INDEXED-SORT.

The current implementation of sort-item-by-price has something like the following:

sort-items-by-price (items nilgreater)

INDEXED-SORT var1 var1
             var2 var2
             getindex WITH-OBJECT var1
                        ELEMENT position 0
                                sequence @price
             sequence items
  CALL :<=>-nil.
    var1
    var2
    nilgreater

To reverse the sort you would would switch the arguments and the boolean in the call to <=>-nil. — Something like:

sort-items-by-price (items nilgreater)

INDEXED-SORT var1 var1
             var2 var2
             getindex WITH-OBJECT var1
                        ELEMENT position 0
                                sequence @price
             sequence items
  CALL :<=>-nil.
    var2
    var1
    NOT nilgreater

I hope this helps!–SCR

RTML Tips# 1—Optimizing Publishing Times

The following post comes from Sheridan Rawlins, Yahoo! Small Business engineer, covers the use of RTML Operators and how certain custom template configurations can lead to long publishing times. The post is intended for Yahoo! Store Developers or merchants with an advanced understanding of RTML. Learn more about RTML.

In my role as lead engineer for the Editor & RTML language, I have come across questions regarding “slow publishing”. Almost invariably, the store or developer requesting assistance has customized their templates in some way. I would like to share some insights by describing answers to some frequently asked questions, along with their resolutions. With this knowledge, developers should be able to spread the best-practices for reducing publishing times for templates.

What causes Editor publishing to take a long time?

The most common root cause of slowness is due to having to fetch objects from disk. We do our best to cache as much as we can in memory, but as complex RTML templates are run on larger stores, or stores with complex object relationships, objects may be written back to disk and flushed from the cache to make room for other objects.

What operators cause objects to be read from disk?

• WITH-OBJECT
• FOR-EACH-OBJECT
• GET-PATH-TO (single “breadcrumb”)
• GET-ALL-PATHS-TO (multi “breadcrumbs”)

Contrary to popular belief, the WHOLE-CONTENTS operator does not cause cache-misses. However, it is usually used in conjunction with a FOR-EACH-OBJECT loop, which causes every object to be read into memory sequentially, almost guaranteeing that the cache will overflow and cause disk accesses.

What can I do to increase the speed of publishing my store?

There are several techniques to use to help improve the speed of publishing:

• Don’t repeat yourself – use WITH= to save the results of expensive operations where possible.
• Try not to nest FOR-EACH-OBJECT loops. Sometimes it’s inevitable, due to the nature of what you’re trying to accomplish, just be aware of this, and use trick #1 when you need to get object information (such as @name, or @price) multiple times.
• Try to keep your store hierarchy simple (see below)

I use the same templates for several stores. Why does one store publish faster than another?

Sometimes algorithms can be affected by the data. Take for instance multi-breadcrumbs. We’ve provided the operator GET-ALL-PATHS-TO in order to provide the fastest means of getting paths typically from INDEX -> id. The number of paths returned depends upon the data set.

Figure 1 – single inheritance (1 path per item) 

If all items and sections are only contained by one section or index, then the answer set is small. (See Figure 1)

Figure 2 – multiple inheritance (3 paths)

If, however, a store has items listed in many sections, then the problem grows to be as many sections as each item is listed in. (See Figure 2)

Figure 3 – multiple inheritance at multiple levels (9 paths)

Progressing further, if sections are contained in other sections, and yet all paths lead upwards, the problem multiplies to be all combinations of sections leading up to the top element. (See Figure 3)

Figure 4 – multiple inheritance with loops (585 paths)

The highest level of complexity is when there are “loops” in the hierarchy coupled with multiple sections leading to INDEX. If section A is contained in B, and B is contained in A, then the problem blows into the most complex yet, as it is each combination with 2 elements * each combination with 3 elements, etc up to the number of elements which are cross-referential. (See Figure 4)

This complexity also changes drastically with additional sections:

Hopefully these tips will help you make changes to your templates to make them more efficient and quicker to publish. Stay tuned for more RTML tips in later posts.

Sheridan Rawlins
Yahoo! Small Business