The complexity of using forwarding addresses for decentralized object finding

We inves t iga te the complex i ty of using forwarding addresses as a mechan i sm for f inding highly mobile objects in a decen t ra l i zed d i s t r i bu t ed sys tem. We analyze the cost of using each of three objec t finding pro tocols base on forward ing addresses, der iving ammor t i zed wors t case uppe r and lower bounds as well as e s t ima tes for the average case cost . The average case analyses of the two s impler pro tocols hre done by represen t ing the s ta te of the sys tem using Markov chains. In add i t i on to the app l ica t ion at hand, these techniques y ie ld a new and more accura te me thod for der iv ing the d i s t r i bu t ion of the dep ths ver t ices in r andom, roo ted , l abe led trees. The worst case cost per access of a p a t h compress ing p ro toco l is O(logN/log(a/rn)), where N is the number of distinct processors the object visits in its lifetime, a is the number of times it is accessed, and rn is the number of times it moves. The technique used in this analysis also yields an improved upper bound on cost of using a well-known sub-optimal algorithm for the set union problem. I 1 Decentralized Object Finding. We are in te res ted in p rob lems t ha t arise in the imp lemen ta t ion of large, high per formance , d i s t r ibu ted , 1This work was begun while the author held an IBM Gradu a t e Fel lowship. It continued while the author was supported under DARPA Contract MDA-903-82-C-0424 as well as NSP Grants MCS-8004111 and DCR-8402565. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specfic permission. o b j e c t o r i e n t e d c o m p u t e r sys tems. In this pape r we address the p rob lem of ensur ing t ha t the processors in such a sy s t em will be able to find and therefore remote ly access all of the ob jec t s to which they can refer. This object finding problem is not very interest ing when all of the r emote ly accessible objects in the sys tem are immobi le . I t becomes more in teres t ing when these objec ts become mobile as in the Eden sys tem [ABLN83,Bla85,LLA*81], and more chal lenging when, as ill Emera ld [Bt lJL85,Hut85] , i t becomes possible to move several r emote ly accessible objects in a single in t e r -p rocesso r message as pa r t of a r emote invocat ion mechanism. Such a sys tem will, when scaled up to a large size, exhib i t massive paral le l ism. Each processor will be c rea t ing and des t roy ing smal l objects at a high rate . Some objects wil l be h ighly mobi le , and there will be a lot of in te rac t ion be tween processors in the form of moving and accessing these mobi le objects . We expec t t ha t such a sys tem will also exhibi t good local i ty proper t ies . Most of the objec ts wil l spend thei r lives on one processor and be accessed only by t ha t processor. Of the objects t ha t move, most of those will be cons t ra ined to move among a smal l number of processors and will be accessed only by those processors . I t is an i m p o r t a n t goal to use preserve the independence and the pe r fo rmance of au tonomous compu ta t ions. If an object moves be tween and is accessed by only two processors then i t makes l i t t le sense to force those processors to use the resources of a th i rd to keep t r ack of the loca t ion of the object . In t roduc ing the add i t iona l processors t ha t imp lemen t a s y s t e m provided , cent ra l ized name ( locat ion) service can only increase the expense of using mobi le objects . Fu r the r more, because each processor t h a t provides t ha t service will keep t r ack of many objec t s it is a po t en t i a l bo t t l eneck at which logical ly i ndependen t compu ta t ions unnecessar i ly compe te for resources. In con t ras t , forwarding addresses a p p e a r to be a s imple and n a t u r a l mechan i sm well su i t ed to an © 1 9 8 6 A C M 0 8 9 7 9 1 1 9 8 9 / 8 6 / 0 8 0 0 0 1 0 8 75¢ lo8 object-oriented distributed system. They are especially attractive in that only those processors at which the object has been located will ever be required to participate in a protocol to find it. The dependence of the cost of using forwarding addresses for object finding will be affected by overall system size only to the extent that it affects the size of identifiers and the costs of message routing and forwarding. for maintenance purposes by making it easy to detect forwarding addresses that are seriously out of date or which refer to objects that no longer exist. A processor that has an identifier for object X can initiate a remote access of X by dispatching a request message along a path of forwarding addresses. We impose the following three conditions in order to guarantee that such a path will lead to one of X's laandles. 1.1 Objects and Forwarding Addresses. Our definition of an object conforms to our intuitive notion of physical objects. The state of an object is represented by one or more data structures called handles, each of which is located on a distinct processor. If an object always has exactly one handle then we say that it is atomic. Otherwise it is distributed. An object can be moved only by an operation that explicitly transfers a handle from one processor to another. An object is found when an access request message from a processor originating a request is delivered to a processor at which one of the object's handles is located. Once this message has been delivered to a handle it is the responsibility of the object itself to use an appropriate protocol to ensure that it deals with requests in a consistent and reliable way [Tho79,Gif79,Lam78]. We assume that the underlying system implements a complete logical communication network such that any processor can send a message directly to any other without worrying about routing. A forwarding address for object X is a data structure that contains at least the following three items. 1. An identifier for X. The choice of identifiers must be such that it is easy to decide whether a pair of forwarding addresses both refer to a single object. 2. A timestamp that indicates tile age of this forwarding address relative to other forwarding addresses for the same object. 3. A location, tile set of identifiers of the processors at which X's handles were located when the first copy of this forwarding address was created. Because the timestamp within a forwarding address is used only to check its age relative to other forwarding addresses for the same object, an adequate implementation of the clock is to keep a counter of the number of times attempts have been made to move the object. This counter can be kept as part of the object's state. On the other hand, using a clock with a closer correspondence to physical time can be useful 1. Whenever an identifier that carrys with it the right to access an object remotely is passed between processors it must be encapsulated within an appropriate forwarding address. Initially the only processors to have such an identifier are those in the object's initial location. Thus every processor with the right of remote access will have been sent a forwarding address. 2. Whenever any of an object's handles is moved, a forwarding address must be left behind. A new forwarding address naming the new location the object is created and it must be left at all of the processors taking part in the move operation. 3. A processor p is allowed to discard a forwarding address only if a) it has been replaced with a newer forwarding address for the same object, b} p knows that the object no longer exists, or c) p knows that it will never need to reference the object in the future and that no other processor has a forwarding address pointing to p. We can interpret the forwarding addresses for an object as defining a directed graph with the processors as vertices and with an edge labeled with timestamp T from processor p to processor q if and only if p has a forwarding address created at T that points to q. These three conditions guarantee that the set of processors that can refer to the object will be in a single connected component and that the timestamp labels along all paths from any processor not holding a handle will increase monotonically until a handle is reached. The only strongly connected component of the graph will be among the processors currently holding handles. Thus, following any path will lead to a handle. When the object is atomic the graph will be a tree with the edges directed towards the root, the current locaton of the object. Given this basic mechanism we are free to design protocols to use it efficiently and reliably. In particular, in order to keep the paths short a protocol can add additional communication of forwarding addresses beyond the minimum required above. If it is known that one of an object's handles is less mobile than the others then a protocol might use paths to