Remote Collection

Lately I was trying to implement remote collection that supports paging (or more of remote moving window of a data set). If you are lucky enough to use FDS2 than you don't to worry much about paging and can stop reading.

When I started to look into the problem I tried to use ItemPendingError but without much success (mostly because I could not find any decent doc or sample). I still believe that it's the way to go but I found the article on similar topic: http://weblogs.macromedia.com/mchotin/archives/2004/03/large_data_sets.cfm that used somewhat different approach (probably because it's not for Flex2 but Flex1.5) and decided to try it out with Flex2.

If you want to implement collection data provider in Flex2 you need to use either IList or ICollectionView. IList interface is somewhat easier so I decided to go with it.

Below is a sample implementation of IList. It's based on Matt Chotin's article I mentioned above. The main idea the same, when getItemAt() method is called by DataGrid, check if row is local, if not download it and send event notification. I also added some code to remove previously loaded item (we keep locally up to 3 pages and as current window of data moves, old data gets discarded)

package
{
 import flash.events.*;
 import flash.utils.Timer;

 import mx.collections.ArrayCollection;
 import mx.collections.ICollectionView;
 import mx.collections.IList;
 import mx.collections.IViewCursor;
 import mx.collections.Sort;
 import mx.collections.errors.ItemPendingError;
 import mx.events.*;
 import mx.controls.DataGrid;

 public class RemoteList extends EventDispatcher implements IList
 {
  public var grid:DataGrid;

  private var remoteDataSet:IRemoteDataSet;
  private var localData : Array;
  private var pagesLoaded : Array;
  private var pagesPending : Array;
  public var  pageSize : int;
  private var isDataInit:Boolean = false;

  private var eventDispatcher:EventDispatcher; 


  public function RemoteList(ds:IRemoteDataSet)
  {
   localData = new Array();
   pagesLoaded = new Array();
   pagesPending = new Array();
   pageSize = 10; //default

   remoteDataSet = ds;
   ds.registerListeners("getRows", onDataReady, onError);
   ds.registerListeners("getLength", onLengthReady, onError);
   remoteDataSet.getLength();

  }

  private function onLengthReady(result:Object):void
  {
   localData.length = uint(result.result);
   trace("onLengthReady:" + localData.length)
   dispatchRefreshEvent();
  }

  public function get length():int
  {
//   trace("length:" + localData.length);
   return localData.length;
  }

  public function getItemAt(index:int, prefetch:int=0.0):Object
  {
//   trace("getItemAt:" + index);
   var item:Object = localData[index];
   if (item == null)
   {
    item = miss(index);
   }
   return item;
  }

  public function getItemIndex(item:Object):int
  {
   trace("getItemIndex:" + item);
   return -1;
  }

  private function miss(index : Number) : Object
  {
   var page : Number = Math.floor(index / pageSize);

   //it's possible that the page is already being loaded
   if (pagesPending[page] == true)
   {
    //this miss event is useful for just monitoring what's going on
    //dispatchEvent({type: "miss", index: index, alreadyPending: true});
    return {id:"Loading:" + index};
   }

   //if the page is not loaded call for it
   //var call = localDataService.getElements(page * pageSize, pageSize, this);
   trace("remoteDataSet.loadData: " + page);
   remoteDataSet.loadData(page, pageSize);

   pagesPending[page] = true;
   //dispatchEvent({type: "miss", index: index, alreadyPending: false});
   return {id:"XXXLoading:" + index};;
  }

  public function onDataReady(loadedData:Object):void
  {
   var result:Array = loadedData.result[0]["value"].toArray();
   var page:int = loadedData.result[1]["value"];

   pagesPending[page] = false;
   pagesLoaded[page] = true;
   var beginIdx : Number = page * pageSize;

   localData.splice.apply(localData, [beginIdx, result.length].concat(result));

   cleanInvisiblePages(page);

   trace("localData is updated: " + beginIdx + "-" + (beginIdx + result.length) );            

   if (isDataInit == false) {
    isDataInit = true;
    dispatchResetEvent(); // make columns are initialized
   } else {
    dispatchReplaceEvent(result, beginIdx);         
    //dispatchRefreshEvent();         
   }

   //trace(localData[51]);
  }

  private function cleanInvisiblePages(page:int):void
  {
   trace("start cleaning");
   for (var k:int = 0; k < pagesLoaded.length; k++)
   {
    if ( pagesLoaded[k] == true &&  (k > (page+1)  k < (page-1)) )
    {
     var start:int = k*pageSize;
     var end:int = start + pageSize;
     end = (end > length) ? length : end;
  
     for (var i:int = start; i < end; i++)
     {
      localData[i] = undefined;
     }
     trace("removed: " + start + "-" + end)          
 
     pagesLoaded[k] = false;
     pagesPending[k] = false;
    }
   }
  }

  public function onError(error:Object):void
  {
   trace("error");
  }

  private function dispatchReplaceEvent(items:Array, startIndex:int):void
  {
   trace("dispatchReplaceEvent:" + startIndex );
   var eventCollectionChange:CollectionEvent =
   new CollectionEvent(CollectionEvent.COLLECTION_CHANGE);
   eventCollectionChange.kind = CollectionEventKind.REPLACE;
   eventCollectionChange.items = items;
   eventCollectionChange.location = startIndex;
   var result:Boolean = dispatchEvent(eventCollectionChange);

   grid.rowCount = 100;
   grid.validateNow();
  }

  private function dispatchRefreshEvent():void
  {
   trace("dispatchRefreshEvent");
   var eventCollectionChange:CollectionEvent =
   new CollectionEvent(CollectionEvent.COLLECTION_CHANGE);
   eventCollectionChange.kind = CollectionEventKind.REFRESH;
   var result:Boolean = dispatchEvent(eventCollectionChange);
  }

  private function dispatchResetEvent():void
  {
   trace("dispatchResetEvent");
   var eventCollectionChange:CollectionEvent =
   new CollectionEvent(CollectionEvent.COLLECTION_CHANGE);
   eventCollectionChange.kind = CollectionEventKind.RESET;
   var result:Boolean = dispatchEvent(eventCollectionChange);
  }


  public function addItemAt(item:Object, index:int):void
  {
  }

  public function toArray():Array
  {
   return null;
  }

  public function itemUpdated(item:Object, property:Object=null, oldValue:Object=null, newValue:Object=null):void
  {
  }

  public function removeAll():void
  {
  }

  public function setItemAt(item:Object, index:int):Object
  {
   return null;
  }

  public function removeItemAt(index:int):Object
  {
   return null;
  }

  public function addItem(item:Object):void
  {
  }
 }
}

To get actual data we delegate request to IRemoteDataSet interface that has several methods to load data and data length:

package
{
 public interface IRemoteDataSet
 {
  function registerListeners(method:String, dataHandler:Function, faultHandler:Function):void;

  function getLength():void;

  function loadData(startIndex:int, size:int):void;

  function sort(param:Object):void;

 }
}

Below is a sample of implementation of the interface (this is just a mock implementation that uses Timer to simulate data loading delay):

package
{
 import flash.utils.Timer;
 import flash.events.TimerEvent;
 import mx.collections.ArrayCollection;
 import mx.utils.ObjectProxy;

 public class RemoteDataSet implements IRemoteDataSet
 {
  private var len:uint = 42;
  private var handlerMap:Object = new Object();

  public function loadData(page:int, size:int):void
  {
   // create data
   var arr:Array = [];
   var startIndex:int = page * size;
   var endIndex:int = startIndex + size;
   var max:int = (endIndex > len) ? len : endIndex;
 
   for (var i:int=startIndex; i < max; i++)
   {
    var obj:Object = createObject();
    obj["id"] = i;
    obj["name"] = "name" + i;
    arr.push(new ObjectProxy(obj));
   }
 
   // simulate remote connection with 2 sec delay
   var timer:Timer = new Timer(2000, 1);
   timer.addEventListener(TimerEvent.TIMER, function ():void
    {
     handlerMap["getRows"]["dataHandler"].call(this, {result:[{value:new ArrayCollection(arr)}, {value:page}]}); 
    });
   timer.start();
  }

  public function registerListeners(method:String, dataHandler:Function, faultHandler:Function):void
  {
   handlerMap[method] = new Object();
   handlerMap[method]["dataHandler"] = dataHandler;
   handlerMap[method]["faultHandler"] = faultHandler;
  }

  private function createObject():Object
  {
   var result:Object = new Object();
   return result;
  }

  public function sort(param:Object):void
  {
  }

  public function getLength():void
  {
   handlerMap["getLength"]["dataHandler"].call(this, {result:len});
  }

 }
}

So far so good, but unfortunatly I encountered an issue (maybe bug in DataGrid or my lack of Flex knowledge) - sometimes if you scroll up or down, the first row of the grid would not be redrawn (meaning it would display temp data like "Loading") despite the fact that actual data source is up to date. If you force refresh (by changing size of browser window) the correct data would appear.

If somebody knows what's going or if there are better ways to achieve paging please share the knowledge.

Here is the link to the whole project with demo and another sample of RemoteDataSet that uses web service (server side web services code are not included).