Archival Appraisal and Collection:

Issues, Challenges, New Approaches

Special Lecture Series

University of Maryland and to NARA Staff

NARA 2 Auditorium: College Park, Md., USA

21-22 April 1999

By TERRY COOK

Archival Studies

University of Manitoba

©Copyright by Terry Cook 1999

Good morning, and thank you for coming. I appreciate being welcomed here at NARA, although fear that I cannot live up to the expectations raised by the flattering notices of me, either beforehand or today.

This is the third of four public lectures sponsored by the Archival Studies programme of the University of Maryland, and I do thank Bruce Dearstyne and Chris Halonen for having me, and for their efficient managing of the arrangements that got me here. I think that their revitalized programme is an important part of the North American archival education scene, especially so with the possibilities of fruitful interaction with working professionals by its close proximity to NARA, and I so am pleased to support it this way. I know that at the National Archives of Canada (NAC), we would have been delighted to have such a programmes, with its wealth of bright students to tap into, sitting on our doorstep.

I will confess that I stand before you this morning a little nervous, even if honoured to be here. First, to repeat words I used in my opening address Monday afternoon, I have been a long-time admirer of many aspects and people of the National Archives and Records Administration, in whose presence I now stand, apparently with the chore to try to sound wise to those very people who have been the fount of much wisdom for me over the years. I have visited NARA for two, intensive, multi-day tours in the past, as well as on several shorter occasions, and in turn have co-hosted at the National Archives of Canada two intensive visits in return by NARA specialists. As a consequence, a good deal of my work in appraisal and electronic records at the NAC and in my professional writing has been influenced by ideas and practices pioneered here. If I may say so, I still think the Justice Department Litigation Case File appraisal report is one of the finest piece of archival analysis ever produced anywhere, and I was exposed to it at a formative moment in my own thinking. It is one example of that “free trade in archival ideas” that National Archivist Jean-Pierre Wallot referred to between the two institutions and countries when he spoke in this room a few years ago.

And the second reason for some nervousness is that I stand in the home of Theodore R. Schellenberg, the world’s best known and most influential archival theorist, whose work certainly solidly grounded my own transformation from historian to archivist in the mid-1970's. In a recent analysis I published on the history of archival ideas in this century, I devoted more space to Schellenberg than to any other writer or group of writers -- 50% more space than to his nearest rivals: Jenkinson and the Dutch trio of Muller, Feith, and Fruin. As a long-time admirer of Schellenberg, if in some aspects not an uncritical one, I find it difficult to conceive that I should be talking about appraisal here in his home when, for over two decades, I have found inspiration from his ideas and methodologies. I approach his legacy with respect.

*************************

I’ve been asked to speak this morning about the issues, challenges, and new approaches to archival appraisal. I would like to organize my remarks in two sections: the first on the theoretical or philosophical underpinnings of determining “value,” which to me is an overlooked but essential dimension of appraisal, and the second part on outlining an approach to appraisal and records disposition that we have implemented at the National Archives of Canada, and which some other countries and jurisdictions have found of interest and worth imitating. Perhaps there will be some relevant resonances here, for you, from this work, as NARA itself begins a business process re-examination of its appraisal and disposition work. Certainly Governor Carlin’s 1997 strategic plan, which envisions a functions-based appraisal approach, is very compatible to what has been done at the NAC. But the functions-based approach to appraisal and disposition at the practical level of operational reality must reflect a defendable set of theories or concepts of why some records are important and some are not. Unless one can explain to our clients and publics, with conviction and convincingly, that we have a theoretical vision of what makes some records valuable and most others not, the best, most efficient methodologies in the world will count for nothing. This morning I am offering one person’s reading of the issue, which as I’ve just said, has found positive responses in some countries, but also polite demurring in others; I do so with the recognition that different jurisdictions, different traditions, and differing political realities will modify the applicability of what works in Canada in other situations.

****************************

Appraisal imposes a heavy social responsibility on archivists. In the stirring words of Pam Wernich, a South African archivist writing in 1988, archivists are doing nothing less than "moulding the future of our documentary heritage." Archivists determine "which elements of social life are imparted to future generations...." As a profession, we archivists need to realize continually the gravity of this task. We are literally creating archives. We are deciding what is remembered and what is forgotten, who in society is visible and who remains invisible, who has a voice and who does not. In this act of creation, we must remain extraordinarily sensitive to the political, social, and philosophical nature of documents individually, of archives collectively, of archival functions, of archivists’ personal bias, and most especially of archival appraisal, for that process defines the creators, functions, and activities to be reflected in archives, by defining, choosing, selecting which related documents are to be preserved permanently, and thus are to enjoy all subsequent archival processes (description, conservation, exhibition, reference, etc.), and, as starkly, and with finality, which are destroyed, excluded from archives, forgotten from memory. In many societies, and I’ll mention this in more detail tomorrow, certain classes, regions, ethnic groups, or races, women as a gender and non-heterosexual people, have been de-legitimized by their relative or absolute exclusion from archives, and thus from history and mythology -- sometimes unconsciously and carelessly, sometimes consciously and deliberately. Why? And is that the kind of restrictive archival legacy we want to bequeath for our own times to future generations? This is not so much the “Archivist as God” syndrome as it is “Archivist as Society’s Conscience and Mirror,” as its professional representative entrusted with creating an archival legacy.

Pam Wernich further wrote that "the selection of records for preservation from the vast quagmire of official documentation represents the greatest professional challenge and the most important area of archival activity." I agree with her, with the qualification that the identification and selection of records to form our archival heritage from the private non-government sphere of human activity is equally challenging, but that aspect of the appraisal function, in consideration of my audience and time constraints, I will place aside for today. Helen Samuels and Richard Cox have called appraisal the archivist’s “first responsibility” from which all else flows. Surprisingly, however, it is not generally a responsibility well met. Richard Berner dedicated his important 1983 book, Archival Theory and Practice in the United States, curiously, in light of what I’m about to say, to his “friend and teacher,” T.R. Schellenberg. Yet Berner stated that there was no appraisal theory covered in his book on archival theory, noting that “the main reason for this omission is the primitive nature of its development.” And that in a book dedicated to Schellenberg, who has been called, with much justification, "the father of appraisal theory in the United States, ” and, I would add, the world! Berner notes that most so-called theory has “not yet moved significantly beyond the taxonomic stage in dealing with appraisal.” That he defines as the mere naming or classification of values, and as merely “one of the first stages of analysis in the descriptive sciences.” Berner concludes -- which is not a bad theme for part of my address -- that “a body of appraisal theory is perhaps the most pressing need in the archival field today. If we delay much longer,” he warns, with some prescience, considering remarks quoted in the recent New Yorker article from some NARA staff, “we will all be smothered by useless paper and all forms of machine-readable records, unable to distinguish those worth saving from all the rest.” While theory about appraisal “is still in early gestation,” in his words, theories about “arrangement and description have emerged from a protracted pregnancy, and have a coherence now that is lacking in appraisal practices.” While some writing on appraisal has certainly occurred since Berner wrote this in 1983, most of that writing is strategic, not theoretical, and no consensus has emerged.

As Berner implied, the most difficult, and the most overlooked, dimension of appraisal is its theoretical core upon which the strategy and methodology of appraisal practice depends, or should depend. Before discussing appraisal theory, however, it may be useful to explore the difference between "archival theory" generally, and "appraisal theory." Archival theory is derived from the characteristics of records and the context of their creation and contemporary organization and use. Beginning with the French articulation of respect des fonds in the nineteenth century, and reinforced a century ago by the famous Dutch Manual of 1898, and in subsequent pioneering books by Jenkinson and Casanova before 1930, classic archival theory focused on the organic character and evidential properties of records as being the very core of archival thinking. Archives were viewed in such early thinking as the un-self-conscious and thus natural byproducts of administrative or human activity, and if so maintained, without tampering and in unbroken or traceable custody, they could be considered reliable and authentic evidence of the actions of those who created them. Concern for records having this quality of evidence is not something new that electronic records has imposed on the profession, therefore, but rather the heart of traditional archival theory. Such an initial theoretical focus on respecting the original order and the context of creation reflected those pioneering archivists’ preoccupation with arranging and describing older records of uncertain provenance. But such archival theory, which concerns the nature of records, has no direct benefit to appraisal theory, which concerns the value of records, which are the reasons or principles upon which some records are judged to be important and some are not. Of course, if records are not authentic or reliable, if they do not have the characteristics of evidence, their value is very much diminished, but that is true of all records: the letter of a cabinet secretary and the invoice ordering new pencils. And as I suggested yesterday, it holds true whether the letter or invoice is in paper or electronic form.

The inherent nature of records, as the natural, organic by-products of their creators' actions, does not help determine which records, of the billions and billions created each year, indeed, each day, and each hour, actually have long-term or archival value. All records by definition bear evidence -- sometimes imperfectly, it is true -- of the acts and transactions that created them, and all have (or had) an original order and context. That being said, one is not left any further ahead in defining issues of value, or importance, or significance, or any of the other terms used in appraisal to distinguish WHY some records are kept and others destroyed. What differs and is important, is not the evidential and contextual nature of the record, but rather the various and differing contexts of the acts and transactions, or at a higher level, of the functions and programmes, that caused the record to be created. To return to my example, in the office of a cabinet secretary, the secret negotiating of a new trade agreement with China and the ordering of pencils are both business transactions that lead to the creation of documents, maybe even under ideal management, in systems implementing the full functional requirements for record-keeping, but clearly one function generates records of long-term importance and the other does not. That in a nutshell is the justification for the functional appraisal that I have been developing in recent years, and which the National Archives of Canada has formally implemented -- but I will come back to that. When Governor Carlin refers to ready access to “essential evidence” as the central mission of NARA, the focus of appraisal theory is determining what is “essential,” while the focus of archival theory is defining record-keeping standards that will produce “evidence.”

Appraisal theory, then, to offer a definition, articulates concepts that determine "value," and enunciates the generic attributes of those concepts that apply to the selection of records for enduring preservation. Unfortunately, appraisal theories of “value,” which in many ways reflect such classic (and difficult) philosophical questions as what is the good and what is not, what is beautiful and what is not, have rarely been articulated within the archival profession -- as Berner noted. Archivists seem to assume that records contain inherent or self-evident informational, evidential, or legal values, which the archivist need only recognize, or judge against a circular, taxonomic set of criteria, and then act accordingly to preserve the records having those values. That kind of recognition and action is, in my opinion, strategic or methodological or procedural, however, not theoretical. Theory tries to answer why some records are archival and others are not; strategy and methodology answers how, where, when, and with whom such a determination should occur.

Now there is a healthy scepticism in the archival profession about theory, which can become a kind of arid neo-Scholasticism of debating how many archivists can dance inside a Hollinger box. I think from my travels that I could observe that this scepticism towards theory is most pronounced here in the United States, perhaps fittingly so as the home of James and Dewey’s pragmatism, of American know-how, of rolling up the sleeves and getting the job done. I have some sympathy with that view, especially when theory seems an external imposition taking little cognizance of workplace realities. I worked for 23 years in a national archives where producing hard, real, practical results was what counted — not endless speculation of why we were doing the work, but getting on with it. Yet when the work needs to be re-conceptualized, when new factors arise that cause accepted strategies and methodologies to break down, then theory can provide the basic principles for restructuring or reengineering processes, it can focus the justifications necessary to explain why we do what we do to our various publics, and it can animate a vision necessary to unite staff behind new approaches. It is important to remember that the opposite of practical is impractical, not theoretical. Theory is rather the complement to practice, and theory and practice should interact and cross-fertilize each other, rather than one being derivative of or dependent on the other.

As far as I can determine, archivists, when they have addressed appraisal theory, have so far suggested that degrees of "value," or "importance," or "significance," that should underpin appraisal strategy and actual appraisal decisions, can only come from, or be imposed by, one of three sources: the creator, the user, or society at large. Other possibilities may exist, but they have not yet been developed into appraisal theory. If any of you this morning can think of any others, I would be pleased to hear of them. Let us look briefly at these three sources of determining value, which amount to three theories of appraisal, although they may not have been consciously articulated that way by their authors. All attempt to justify WHY some records have long-term value. Until that justification is clear in our minds, there is little point, I believe, in building appraisal strategies, establishing improved or more efficient disposition regimes, or articulating functional or other appraisal criteria, for these are all designed, by definition, to identify and protect records having long-term value. But one has to know first what makes something have “value,” what one is looking for, before developing better means to find it. Theory comes first. Theory sets forth the principles upon which we must agree to proceed. Theory allows us to defend our choices to contemporary critics and to posterity.

The first appraisal theory model is allowing the creator to determine “value” and therefore to make the archival appraisal decision. This was the approach advocated by Sir Hilary Jenkinson, and still finds some advocates. This approach has the advantage of allowing those closest to the records, and to the functions and activities that generated them, to isolate the best records reflecting those activities, to undertake a natural winnowing or reduction of the bulk of records over time so that only the essential core or bare minimum remains. The assumption is that the original actors in the events and issues are best qualified to do this selection because they know the issues. Thus, a natural residue will emerge, which in the fulness of time the archivist will take into custody and preserve forever. This approach assumes that the creating institution is relatively stable, small in scale, and straight-forward in its functions and activities, that actions and events are institution-specific and narrowly and cleanly focused within a Weberian vertical or hierarchical bureaucracy, and that these characteristics -- small, stable, focused, centralized -- are also replicated in the institution’s recording technologies and record-keeping systems, and in its highly educated staff as in Jenkinson’s Oxbridge days. Alas, dear colleagues, none of these conditions pertain in the vast majority of late twentieth-century institutional record creators.

Unfortunately, even if these conditions were to exist, this passive approach also sanctions the destruction of archivally valuable records for any reason the creator, or subsequent owner, or controller, may determine, from concern over personal embarrassment or scandal, to over-zealous protection of privacy, to thwarting openness and accountability in government, on to "politically correct" or symbolic acts used to justify the present by destroying the past, as when the French Revolutionists deliberately destroyed the mediaeval and royalist documentary legacy of France or -- from another angle but showing the same motivation -- when modern military strategists deliberately bombed the archives and libraries of the Balkans as a means to break the spirit of opponents by destroying their roots and memories. That this creator-driven approach to determining value might lead to serious abuses, thus undermining the accountability of government to the governed, outside the dramatic intensities of war and revolution, has also been graphically revealed in recent years in Canada over illegal records destruction relating to the HIV-tainted blood scandal and our tarnishing peace-keeping murders in Somalia, in the United States regarding the Nixon Watergate tape recordings and Bush White House e-mail records, in Australia over child abuse records at the John Oxley Centre, and massively in South Africa by apartheid officials as sworn testimony before the Truth and Reconciliation Commission has detailed. These are the most famous, but by no means not the only cases of illegal records destruction.

Even putting such abuses aside, allowing the creator to determine “value” privileges the powerful and the institutional in society those who have the resources and infrastructure to create and manage records in an orderly way, who can afford to allow for a natural residue to form and survive over time, and then at the very end of the life cycle, pass these accumulations to an archives for preservation. And most evidently, this approach confuses archival theory and appraisal theory. Archival theory traditionally saw the archivist preserving original order and context, as a kind of invisible guardian who did not interfere with orders, context, values, that were, so it was thought, original, natural, inherited, given. Appraisal by contrast is about “Making Choices,” in the phrase of the National Archives of Australia’s new strategic plan in this area. It is thus also about very active interference to isolate a tiny portion from the whole for long-term preservation. Here, as always, Jenkinson was consistent, recognizing that the logic of archival theory, strictly interpreted, meant that appraisal of records by the archivist was not an appropriate activity. If archives were the organic emanation and residue of documents from a records creator, then severing any records from that organic whole seemed to him to violate a fundamental principle of archival theory. The exercise of "personal judgement" by the archivist, as Jenkinson knew any appraisal must necessarily involve, would tarnish, in his view, the impartiality of archives as evidence, as of course would any consideration of saving archives to meet actual or anticipated uses of records by historians or other researchers. The archivist's role was to keep, not create archives. Some European archival thinkers remain distinctly uncomfortable with the seeming gulf between the alleged objectivity of archival theory (or “archival science”) on the one hand, and, on the other, the evident subjectivity of appraisal theory. I will address this directly tomorrow.

The second approach to appraisal theory assigns “value” according to user needs (actual or anticipated). Best argued by Theodore Schellenberg and holding sway over most of the archival world in the second half of the twentieth century, this approach sought, quite admirably, to broaden the institutional bias of Jenkinson by considering a much wider range of records and researchers’ needs. This theory is grounded in an empirical approach to determining value. If a researcher can use the record, then it has value. If it is hard to identify or to imagine or, increasingly in resource-hard times, to demonstrate use, then the record does not have value. Schellenberg articulated various categories of use, with which we have been familiar since we were baby archivists: primary and secondary uses, and within the latter category, evidential, legal, fiscal, and especially informational research values. Certainly consistent with his focus on society and secondary research, Schellenberg, to his considerable credit, attempted much more than had the Dutch, Jenkinson, or other European authors to build bridges between archivists and librarians, and between archivists caring for institutional records and those responsible for private manuscripts. We should recognize, too, that there are reflections of a broader societal perspective in aspects of Schellenberg, although only indirectly through the filter of history and historiography. If the researchers found uses for records, then it may be assumed that the records must fill needs that society felt, and, again, historians would in their choice of topics and themes would indirectly reflect as well trends in society.

This empirical or pragmatic approach based on use to determining archival value was thus an important step forward, but it may leave archives subject to the loudest lobbying groups of researchers or the latest trends of history graduate schools that either form archival mindsets directly, or produce professors and graduate students who pressure archivists to acquire “relevant” records for their research. Moreover, archivists trained in history or closely related social sciences in their undergraduate and graduate degrees are unlikely to be able to judge empirical use values for the physical sciences, medical, environmental or planetary needs, geographical, aesthetic, or many other fields of human endeavour. Reflecting on several decades of such use-based approaches to defining archives, Gerald Ham has argued that the results are "a selection process so random, so fragmented, so uncoordinated, and even so often accidental...." It could hardly be otherwise, he asserts, for archivists’ value-formation processes became "too closely tied to the ... academic marketplace," with the ultimate result "that archival holdings too often reflected narrow research interests rather than the broad spectrum of human experience. If we cannot transcend these obstacles," Ham continues, "then the archivist will remain at best nothing more than a weathervane moved by the changing winds of historiography." Building teams of archivists and researchers from various disciplines, as advocated by Helen Samuels and later others in the documentation strategy, was the first attempt to transcend these obstacles that Ham raised, but only succeeded by creating others. In any use-driven approach to “value,” the archivist must remain ultimately a prophet trying to predict future research trends rather than an analyst trying to reflect the functions, programmes, and activities of records creators and the broader society in which those creators live, work, play, think, and dream. Use-driven archival paradigms impose criteria on appraisal that are external to the record’s context and thus undermine its provenance, thereby detracting from its role in cultural memory. These are its theoretical weaknesses; I will return to strategic ones later.

While "value" can certainly be defined through the needs, prejudices, and societal influence of Jenkinson’s creators or Schellenberg’s users, I submit that these are not archival values. In both cases, appraisal has been taken from the domain and professional competence of the archivist, who then is left to interpret and implement the wishes of others, whether creators or users. Archivists in these approaches then build strategies and develop criteria to meet these wishes, but they do not articulate appraisal theory. I consequently have some problem with this. Let me say why.

Except in private business corporations, at least directly, archivists are usually perceived, mandated, and paid as society's guardians of its collective memory, its heritage, its past, its history, and thus they have, in my opinion, an obligation to reflect in archives, in each generation, the values of that society that entrusts to them with the professional role of not just keeping archives, but of identifying, selecting, appraising, choosing archives. The third theoretical basis for appraisal, then, is founded on discerning directly the values and trends of the society contemporary to the records’ creation, and translating these into appraisal strategies and methodologies.

Sociologists have suggested that all societies (including the archivists residing in them) assign greater or lesser value to different dimensions of the three-way interplay of social structures, societal functions, and citizens and groups. This is how society functions. Such value assignment to particular functional phenomena will in turn determine, when this insight is transformed into an appraisal model, which related records are declared to be archival or which are not. The appraisal theory I am outlining suggests that such societal values may be determined by the archivist by specifying the generic functional attributes, and points of special intersection or conflict, between the creators of records (that is, structures, agencies, actors); socio-historical trends and patterns (that is, functions, programmes, activities); and clients, customers, citizens, or groups upon whom both function and structure impinge, and whom in turn influence both, directly or indirectly, explicitly or implicitly. Archival appraisal theory in this third option explores the nature of these agents and acts, and the interconnections or interrelationships between them, and assigns greater importance, or "value" to certain functional-structural factors as compared to others. This is why it is known as “functional appraisal.” Because, as Hans Booms noted, it is the functional context of creation and contemporary use that determines value, it is a provenance-based approach to appraisal. Because it looks first at functions rather than records, it has been called a “top down” approach. Because appraisal has traditionally been focused on the value of records, this approach, which focuses first the value of functions, has been called “macroappraisal.” To get people thinking, or just to stir up trouble, I have often said of this third approach to value determination that, while archivists appraise records for eventual use by researchers, archivists should, in the first instance, neither appraise records nor try to anticipate their use. But here I drift from theory into strategy, so let’s define that.

Appraisal strategy, as distinct from appraisal theory that we have just been discussing, provides a way or logic or means or methodology whereby the foregoing theoretical definitions of value may be implemented in working reality. I will not review the appraisal strategy that could be used to implement Jenkinson’s approach to determining value, because, as indicated already, I think his approach is unacceptable, theoretically and morally, for modern archives. It is an irresponsible abdication of the responsibility that society has assigned to archivists. Archivists certainly should consult with creators and investigate their views of what functions and activities might be important, but that advice is just that: advice, to be weighed by the archivist against a much wider range of knowledge from other sources and research. And let me underline that my anti-Jenkinsonian stand has nothing whatsoever to do with two things. First, records management staff in creating agencies very often implement the appraisal decision, even interpret the appraisal criteria, set forward by archivists. That is well and good, and close links along the records continuum or throughout the life cycle are essential in this regard as in others; such staff are archivists’ essential partners and allies. The point is, however, that the major appraisal decisions and criteria are made by the archivist acting on behalf of society, not by agency staff acting on behalf of the creator. And second, being opposed to a Jenkinsonian approach to appraisal theory also has nothing to do with opposing projects like Pittsburgh or UBC, Monash’s SPIRT metadata modelling or Australia continuum record-keeping initiatives, or the National Archives of Canada’s development of functional requirements for record-keeping within an electronic work environment. Those initiatives to ensure the properties of evidence in electronic records -- indeed, to turn data and information into records and thus evidence -- are essential, as we discussed yesterday. But they reflect the concerns of archival theory, not appraisal theory. Remember my point that the cabinet secretary’s policy brief and the invoice for pencils are both records.

For Schellenberg’s theoretical model, or the second, use-based or empirical theoretical appraisal framework, the strategies that have prevailed for the past half century for its implementation are now breaking down. While I have indicated already some theoretical problems with Schellenberg’s approach to determining value, there are practical ones as well. These practical considerations were as significant in the changes implemented at the National Archives in Canada, and elsewhere, as any concern for theoretical clarity, just as it was practical breakdowns in the former records disposition process that forced change, not some new theory of management.

In relying on the archivist’s advanced training in history, combined with input from users groups of historians and other researchers, other specialists, past archival precedents, and informed intuition, used-based values articulated in this way — especially Schellenberg’s central information values — must then be searched for in the records. The central methodological question is a simple one: “Which records are useful?” And in order to allow the historical perspective time to develop and for research needs or historical trends to be manifested, Schellenberg and his colleagues adopted the life-cycle approach to records administration and records disposition: only when records were finished their active and dormant phases of use, long after file closure, would the archival use-based decision be made to keep the record permanently or to destroy it.

In Canada we felt that there were five difficulties with implementing this strategy. First, there are simply far too many records involved to locate these research values with any assurance of accuracy. Assembling large teams, as for the admirable FBI records appraisal case here at NARA in the early 1980's, must remain the rare exception rather than the rule for most archives. Everyone has their favourite figures, and some stark ones were cited in the New Yorker article, such as the volume of cables from the Department of State. My figures are that, if laid end-to-end, the current or active paper records (meaning only those of the most recent five years) of the Government of Canada would circle the globe 144 times or complete 8 trips to the moon and back. To put that into more concrete terms, that is the equivalent of 600,000 full-length books produced per year per archivist -- to appraise. And it is estimated that there are between 100 and 1000 times that "paper" total in all electronic formats. A second problem relates to developing an historical perspective: holding records for 10 or 20 or 30 years after file closure, even 50 or 75 years as in France, until the historical dust settles, as it were, in the life cycle model, to determine historical patterns and trends is not an expenditure of public funds many nations, states, provinces, municipalities, or universities will now tolerate. Thirdly, in the office systems world of the present and future, the appraisal decision needs to be made and coded into the software, at the system design or system remodelling stage, before any record is actually created. Moreover, in that virtual electronic world, as mentioned yesterday, many kinds of records will increasingly not even exist in ways that we traditionally understand in order to be appraised at the record or document level, unless record-keeping regimes are designed to stitch together context, structure, and content into an understandable recorded entity, which requires active “up front” continuum work by archivists, rather than “back end” life cycle analysis.

Fourthly, there is not a single record that I’ve ever met that doesn’t have or suggest some use to someone somewhere. We all have our stories. Within days of destroying scores of boxes invoices and receipts from a larger 400 metres/1300 feet of Department of Fisheries records, 1880-1920, that documented the purchase of boots, rain coats, hats, and other gear for the many fisheries inspectors of Canada, a representative arrived from the Maritime Museum at Lunenbourg, Nova Scotia, who was enthusing that they wanted to put a large mannequin by the front door of a typical fisheries inspector, and were there any records relating to the kind sof clothing, hats, boots, etc., such inspectors might have worn at the turn of the century! Such incidents have often made me think that researchers should not only ask, “Do you have anything on subject x,” but also “Was there ever anything on subject x.” There are simply an unknowable number of possible uses for records, and decisions based on such an approach are often hard to defend as part of a coherent whole to researchers. In the use-based approach, the archivist is put into this impossible accountability position of saying, “I destroyed these records because they were not considered or judged or appraised to be useful,” and to be saying this to a researcher, perhaps by now wide-eyed, foaming at the mouth, and thinking of the letter she or he is going to write to Congress or the New York Times, saying in return, “But I am here to use them. What do you mean they are not useful?” I think that that is an untenable position, politically, in which we have placed ourselves. If judging future possible research uses is a prophet’s game, basing use predictions on past research trends and patterns only has validity if all records, upon which such analysis is performed, are on an equal playing field. Most often they are not, because of the varying difficulties and levels of their arrangement and intellectual access; the quality of their descriptive finding aids; their copyright, FOI, and other legal impediments; their past participation in publications, exhibitions, CD-ROM’s, and web sites — by the archives and others; their being “hot topics” — Gerald Ham’s “weathervane” again — and drawing bursts of interests; the archivists’ own background and thus interest in and promotion of certain records, by word-of-mouth on through reference services to conference presentations and publications. These factors, and a good many others, may hinder or encourage, but they ensure that all records do not have an equal opportunity, or level playing field, to be used. Therefore, conclusions about which records are most used and thus most “valuable” are highly problematic.

Finally, the fifth concern, the life cycle strategy has de facto driven a wedge between records creators and records managers on the one hand, and archivists on the other, although I would assert that this need not necessarily be the case. Records are the purview of records managers; archives are the purview of archivists. This notion which seems natural and normal to North American archivists is quite foreign to Europeans and Australians, and increasingly Canadians. They see more of a common record professional involved in a continuum of record-keeping activities, but, granted, with different perspectives or emphases along that continuum. There is a danger, of course, of too close an identification and affiliation with record creators along the continuum, which could lead back to some of the flaws of the Jenkinsonian model, but the continuum properly managed is an important, and some would say, essential strategic opportunity for archivists, and the life-cycle strategy undercuts it.

So ... too many records, transient and unstable electronic records, an infinite number of possible uses, no luxury of a long passage of time to develop an historical perspective on what might be important or of value, what, then, to do. The response in Canada has been to adopt a top-down rather than a bottom-up strategy for appraisal: one that gives strategic priority to functions and work processes at the top, and not to recorded products or records emerging at the bottom from those functions and processes. I hasten to add that “top-down” relates to a functional decomposition methodology, not to administrative hierarchy or to an assumption that records at the top of the elite carry more “value.” Such functional strategic thinking has long been used in corporate and governmental practices, and is evident today in business system analysis and system design in the world of computers to current interest in business process re‑engineering and government restructuring, as well as increasingly in records management file classification and indexing, and financial and human resource planning and measurement systems. And so, I thought back in 1989, why not also consider its applicability in archival appraisal? Moreover, and especially important, such a strategy is consistent with the third model of theoretical value determination based, as I have noted earlier, on archival records being chosen for best reflecting the sharpest image of the functionality of society. In macroappraisal, there is, then, a congruence of theory and strategy.

It is precisely on this point that the macroappraisal strategic approach has its greatest value. The reasoning behind the macroappraisal approach is simple enough to state. Institutions have certain formal or internally developed functions assigned to them or sanctioned by democratic societies; in this way they are a filter of societal trends, activities, needs, and wishes, of the things and concepts that society “values.” For these assigned functions, the institutions articulate various sub‑functions, which are allocated to different administrative structures or offices, each with a mandate to perform or implement such a function, or part of a function, or perhaps parts of several functions. These offices in turn create various programmes and activities to meet their functional mandates, which in turn lead to specific actions and individual transactions, for the efficient operation or delivery of which information systems are built. Citizens, clients, groups, companies, and associations interact with these functions and structures, programmes and activities, and, depending on the latitude and flexibility allowed for this interaction, they shape, challenge, and modify these programmes in varying degrees. Of all these steps and processes, the record itself is the final evidence within those information systems of all these acts and transactions, and of citizen/societal interactions. This means that the contextual milieu in which records are created ‑‑ what I have called their conceptual rather than their physical provenance ‑‑ is determined by all these factors: functions, sub‑functions, structures, programmes, activities, actions, and transactions, and client interactions, as well as records‑creating processes, systems, and technologies. By focusing archival research on analysing -- that is, “appraising” -- the importance of manageable numbers of these functions, programmes, and activities in the first instance, rather than on appraising billions of records, or tens of thousands of systems, series, and collections, the archivist is able to see the whole forest, rather than just a few trees. Seeing the whole context ultimately means that poorer and duplicate records are more easily identified and eliminated, and that the most succinct, precise, primary record in the best medium for a particular function is more readily targetted (or “appraised”) for archival preservation. In short, macroappraisal strategy shifts the initial and major focus of appraisal from the record to the functional context in which the record is created. Revealing that there is nothing really new in the world, but just new emphases, macroappraisal reflects a three-word insight made a half century ago by American archival pioneer, Margaret Cross Norton: “records follow functions.” Since there are too many records to appraise , why not focus on appraising functions instead, which will then encompass the records that are following them?

Using such knowledge gained by an institutional functional analysis, the main appraisal questions for the archivist are not what has been written (or drawn, photographed, filmed, or automated), where it is, and what research value does it have. Rather, based on this kind of functional-structural decomposition or analysis, the two key appraisal questions are, first, what functions and activities of the creator should be documented (rather than what documentation should be kept?) and, secondly, who, in articulating and implementing the key functions, programmes, and transactions of the institution, would have had cause to create a document, what type of document would it be, and with whom would that corporate person cooperate or interact in either its creation or its later use? These questions beg a third and really the most important question: which records creators or "functions" (rather than which records) have the most importance? And its converse, which I want to underline strongly from my Canadian “total archives” background, despite the overwhelming majority of my written work having dealt with government records: the necessary converse is to also ask, with archival partners: which functions are poorly documented in institutional records and must be complemented or supplemented by private manuscripts, other archival media, oral history projects, and non-archival documentation (publications, "grey literature," buildings, inscriptions, monuments, museum and gallery artifacts, etc.) -- none of the latter necessarily collected by archivists or at least the institutional or corporate archivist. Only after these questions are answered can the archivist target realistically the actual records or series of records likely to have greatest potential archival value.

Once the macroappraisal is completed, and actual series or classes or systems or collections of records are before the archivist for appraisal, traditional appraisal criteria can be applied to the records in question, where greater granularity is necessary or desirable. Such criteria are used to refine further the value of individual records or small groupings or series of records within the theoretical‑strategic functional-structural matrix. Political, technical, legal, and preservation issues are also considered at this point. Known research uses may also be considered, at this final stage only, but not driving the process. If the strategy is called macroappraisal, these record-related criteria are microappraisal. Such microappraisal criteria involve assessing such factors as age, uniqueness, aesthetics, time span, authenticity, completeness, extent, manipulability, fragility, duplication, monetary value, use, etc. Such appraisal criteria are certainly used now in the daily work of archivists and are well articulated in our literature, and thus no more need be said about them here.

Macroappraisal in Canada occurs at three levels, in descending order: first, between all the various institutions or records creators or functions falling under the collection jurisdiction, mandate or acquisition policy of an archives; secondly, between the various functional programmes of a single institution or records creator; and, finally, between the various functions and activities of a single programme or function within that creator -- all using various criteria for assessing the importance of functions to institutions' mandates. For a macroappraisal strategy to work based on functional weighting, we found that a whole new relationship was necessary between the archives and government agencies, to implement what we termed a planned approach to records disposition. This established that records disposition must proceed in a planned, logical manner; that records-creating agencies must be ranked and then approached in priority order, that formal negotiated disposition plans using project management methodology must be in place, and that each appraisal project must be proceed comprehensively across a function, not in small administrative fragments, not medium by medium, not (in the first instance) by looking at a few isolated examples of actual records in a few sub-registries, and not (as is usual) in response to the latest space crisis in some part of the creating department. We also asked some hard questions around basic past assumptions about assigning or approving retention periods for non-archival records and about focusing on passive records destruction approvals rather than active archival targetting.

Macroappraisal strategies and methodologies are increasingly popular, because I believe that they are increasingly necessary. Although reflecting earlier conceptual models by Hans Booms and Helen Samuels, functions-based appraisal was first pioneered, in actual working reality, in Canada as macroappraisal and in the Netherlands as the PIVOT project. Both we and the Dutch felt a little lonely at first, and we were delighted when we discovered each other at the ICA in Montreal in 1992. Both projects are being refreshed at the present time based on lessons learned over the past eight years. The revitalized National Archives of South Africa has formally adopted the Canadian model for its appraisal and disposition work. The National Archives of Australia is now adopting it, as have some Australian and American states and Canadian provinces. Others jurisdictions are exploring its merits. Yet growing popularity doesn’t necessarily mean it is right, or completely right for every national situation. It will be for you to decide whether its approaches and methodologies, which I have just barely summarized this morning, are logical and applicable as NARA undertakes its reengineering project in this area.

For us in Canada, we believe that there is merit in the proposition that for government records, archives should reflect as far as possible the values of the society contemporary to the records creation, and be inclusive of Gerald Ham’s “broad spectrum of human experience.” We believe that the best way to discern society’s values is through an analysis by archivists of functions, structures, and actors directly, and their interaction, rather than being determined indirectly by records creators themselves or by groups of users. And we believe as a corollary, for government archivists especially, but one can transform this to any institutional setting, that the focus of appraisal decision-making should be more on the functions and thus records of governance rather than on those of government. “Governance” is defined to include cognizance of records reflecting the interaction of citizens with the state, the impact of the state on society, and the functions or activities of society itself, as much as it does the records of governing structures and their inward-facing bureaucrats. The records that most sharply reflect that interaction are the targets for archival preservation. And we don’t just believe, but we know that such an approach is only possible based on an extensively re-engineered records disposition relationship between the archives and its client agencies and a renewed commitment that the core of archivists’ work is scholarly research into records, record-keeping systems, and their functional context across space and time.

Thank you.