Keith Instone

I will have a focus on the gaps between research and practice in user experience during my April conference run.

From April 7-10, I will be in Phoenix at the 2010 IA Summit. On Saturday the 10th, I am hosting a discussion on Bridging IA Research and Practice with Andrea Resmini.

We tried to get a pre-conference consortium organized, but will settle for this meeting to hopefully get a head start on 2011. The "research track" at the IA Summit has been a slowly traveled, winding road. One bend was a panel in 2006 and I wondered then if we had turned a corner. Still looking.

From Phoenix, I fly directly to Atlanta for CHI 2010. I am part of a great team that organized a Sunday workshop on Researcher-Practitioner Interaction (the call for participation has passed, but it has a good overview).

I am looking forward to the workshop because I get to spend the day with people who know a lot more than me about the challenges facing the HCI research and practitioner communities. There are a lot of great ideas for closing the gap which have surfaced in their position papers; but there are no easy answers.

If the topic interests you but you are not already part of the workshop, then you can attend our special interest group meeting at 9am on Wednesday, April 14th: How to bring HCI Research and Practice Closer Together. We will recap the workshop and engage more people in the conversation. After it is all done, I hope we will have something substantial to back to the community.

So far, I only know of 1 other person crazy enough to hit the overlapping IA Summit and CHI conference. If you are also one of these crazies, contact me. If you are just a "normal crazy" who is attending only one of these 2 awesome conferences, then be sure to track me down and say hello.

Last month I attended the ASIS&T 2008 annual meeting. I have attended all of the IA Summits (also by ASIS&T) but this was my first "annual meeting." I typed up some notes during the conference, but wanted to wait to compose my thoughts and reflect a little longer. Here ya go.

Highlights for me:

• I liked spending time with colleagues that I have not seen in years. Like: Gary Marchionini and Cathy Marshall, to name just a few. I caught up on their research, they caught up on the challenges for my job.
• I got some IBM work done, meeting David Millen and connecting some of his research to ibm.com initiatives. It still pays to find IBMers by going to (outside) conferences.
• Both plenary sessions helped me think outside my ibm.com box and see how the rest of the world is using the Internet these days. For example, the Digital Youth Project report has now been officially released. Check it out.
• Strengthened/made local connections, such as old colleagues at IAKM and LexisNexis, new ones at OCLC and Wayne State.

So, net, it was worth attending, for sure. Any time I can drive a few hours and hang out with fellow user experience professionals who have traveled here from around the world, I will be there. I do all I can to support UX-related gatherings in the midwest.

What I found really interesting about the event (neither good nor bad, just different), were the elements that made it a meeting first and a conference second. When I compare it with other professional association annual gatherings that I have been to (like the CHI conference by SIGCHI and UPA's conference), then I notice some things that stand out here:

• Business and committee meetings, special interest group planning and other things to "do the business of ASIS&T". These things happen at "conferences" too but they were more prominent here.
• The "intellectually stimulating" content (the conference part) is driven by the ASIS&T org chart. Special interest groups, in this case, sponsor the panels, seminars, etc. In other association events, I think the sessions are more driven by individuals, not "each SIG organizes their own track".
• Lots of fellowship, awards and recognitions. Since this was my first ASIS&T meeting, it sort of felt like my first big reunion with my wife's extended family. I only knew a subset of people, I did not get all of the inside jokes or the personalities, but everyone was very welcoming and wanted me to come back for next year's get-together.

I can see how this type of annual event builds up loyalty. Come to one meeting and you could get "hooked", volunteering for all sorts of worthwhile ASIS&T activities for the next 12 months. You will almost have to attend the following year.

The meeting-focus does provide some challenges for the "technical program" side of the event, however. There is a lot of competition for compelling conference content, and when people ask me "where can I go to really stretch my brain for a week?" then it will be hard to recommend this conference over the many other choices. Making the conference sessions better would also help draw in some "outsiders" (non members), which would hopefully lead to some getting "hooked", and so on.

Which leads me to my only real complaint about the conference. To be blunt: There were too many academics on stage talking for too long. There were not enough researchers from companies on the panels. There were not enough practitioners giving their views. Most sessions did not leave enough time for audience questions and conversations.

I really do not hate academics - I love them, actually. I knew this would meeting would be research and academic focused, I was looking forward to that aspect. I had some great conversations with professors and students (at the SIGUSE symposium, in the hallways, at lunch and at the poster sessions.) But it was too unbalanced for me. If you know me, you know I do not bitch that often, and I only do it because I care and I want to make things better. So let me offer these suggestions for future technical sessions at the annual meeting:

• Each research-oriented panel must have at least 1 member who does not work at a university. A researcher from IBM, Microsoft, some other organization, who talks about the topic from their company's point of view.
• Each panel must have at least 1 practitioner to act as a sounding board. "I hear what you are saying and here is how I deal with it in my world" sort of thing. Get more practitioners up on stage.
• Encourage people to follow "best practices" for presenting. Like: more pictures and less text on slides. Take "clarification" questions during their talk and leave lots of time for discussion after. Provide an overview (only) up front and leave the details for Q&A (so if no one cares about your details, we do not have to hear them).
• When a student is presenting their research, do not allow their advisor on stage. The advisor can only help answer questions after the student has done their best. I do not mind going to a session where students are presenting their work, but I want the students thinking on their feet and answering questions, I do not want their professors explaining things for them.

The research/practitioner divide was exacerbated for me because of the gap between the SIG Information Architecture community and the ASIS&T membership as a whole. That fracture runs deep and goes beyond the ASIS&T annual meeting, so I do not want to get into it here. Those things will get addressed.

I had the honor of presenting one of the keynotes at yesterday's SIG USE annual research symposium (part of the 2008 ASIS&T annual meeting in Columbus). The theme was "Future Directions: Information Behavior in Design & the Making of Relevant Research."

I took on the task of giving SIG USE feedback "from the outside" with these two perspectives:

• Human-computer interaction, information architecture and general user experience professional. What is this thing called "human information behavior (HIB) research" and how does it relate to the research disciplines I am familiar with?
• Practitioner. What can practitioners learn from HIB and apply to their challenges? How do we bridge the research/practice gaps?

I broke my talk down into 3 sections:

• About me and my journey to gain an initial understanding of HIB
• An analysis of the symposium position papers, where I tried to distill them down into both "how do we connect with designers" and the specific research they are doing which I might be able to apply to my "finding information" challenges
• Stories about things I work on for ibm.com, with the hope that they could spur some ideas for some research topics

Download a PDF of my slides (2 meg). I deleted / cleaned up a few things for the public archive. And usual disclaimer: slides geared for the presentation. If you were not there, they may not be very interesting.

I sped through the slides and talked too fast, but I think (hope) that I put forth some good questions for the SIG USE community to debate going forward. The individual discussions and small group work after my talk were very valuable to me. I have some more reading to do (such as information encountering) and contacts of "SIG USE people" who I can stay in touch with. Looking forward to it!

One final note: I can see why SIG USE wins awards from ASIS&T. Very well run.

I will be attending the Collaborative Sensemaking workshop at the HCIL symposium on Friday. I have not been in the "research groove" for about a decade (since I was still at BGSU), but collaborative sensemaking is one research topic that seems to apply to what we are doing on ibm.com and what information architects do every day. So I thought it was worth coming to the workshop to give my practitioner's perspective - and learn more.

I'll report later with more information (after I get back from vacation...).

In the June 2007 Communications of the ACM (Vol 50 #6), Ben Shneiderman has a "Viewpoint" article that hits close to home. "Web science: A provocative invitation to computer science," subtitled "Here's how it can awaken computer science to the interdisciplinary possibilities of the Web's socially embedded computing technology."

I have written about various pieces that Ben mentions (Web science and IA, universal usability, IBM's services science, as examples) but he has tied them together better. And added a wrinkle that I was not concerned with (until now): how to invigorate computer science programs by adopting the Web science framework.

I am not really in touch with the specific woes of computer science, but I can see how the social perspective would make CS research a lot more relevant. Studying social networks instead of computer networks. Researching e-government instead of compilers. Student projects on sharing animation instead of rendering algorithms. Focusing on users instead of computers.

Ben's other main point is that web science can help create a synergy for more interdisciplinary research. Emerging applications like Web 2.0, universal usability and ubiquitous computing are all natural fits under Web science (that traditional computer scientists would likely say are outside their scope).

Ben ends with: "Visionaries say it is time for a change, but will the traditional computer science community accept the invitation? I hope it will."

This CACM article is not online yet but will eventually be in the CACM section of the ACM Digital Library. Here are the references and other mentions from the article while you wait. (Some links lead to summary pages where you need membership to get the full article.)

1. Japan Prize Commemorative Lecture
2. Foundations and trends in web science
3. Creating a science of the web
4. A research manifesto for services science
5. The social life of innovation
6. Crisis and opportunity in computer science (PDF)
7. Leonardo's Laptop
8. 911.gov
9. Web Science Research Initiative

Other reform movements

• Georgia Tech's New Face of Computing
• i-schools

Related to the IA research agenda from the IA Summit, now comes the Web Science Research Initiative with its plans for "web science" and a web research agenda:

There is...a growing realization among many researchers that a clear research agenda aimed at understanding the current, evolving, and potential Web is needed. ...The Web is an engineered space created through formally specified languages and protocols. However, because humans are the creators of Web pages and links between them, their interactions form emergent patterns in the Web at a macroscopic scale. These human interactions are, in turn, governed by social conventions and laws. Web science, therefore, must be inherently interdisciplinary; its goal is to both understand the growth of the Web and to create approaches that allow new powerful and more beneficial patterns to occur.

I know, the web is not IA and IA is not the web, but I see many similarities. For example, from Creating a Science of the Web, I see topics that interest me as an information architect:

• moving from text documents to data resources
• reuse of information
• "policy aware" systems

The Framework for Web Science has more about this research agenda. Where would an IA research agenda overlap, where would it differ?

(Josh has more excerpts, links, and his social web design angle.)

Karl Fast's The Confluence of Research and Practice in Information Architecture is a recap of IA and research from the summit. If you were at the panel discussion you will remember Karl as the guy in the audience who had more / better stuff to say than those of us on the panel.

I like his point about the difference between researchers and academics, and I look forward to the 2007 Summit.

In the week following the IA research panel, I have uncovered a few more things of interest. First and foremost, I found a good place to publish your IA research. Check out the blog for people interested in contributing to the HICSS-40 minitrack on Using Information: New Technologies. The blog is being used to help coordinate the submissions for the conference.

The call for participation lists many beloved IA topics, like social tagging, content analysis, search and access mechanisms, and use of rich media. "Media literacy" - that sounds like a key item to go into the IA Research Agenda.

I think this will be a great venue for IA issues. I must admit, however, that I have never attended HICSS. It is 40 years old, but I have not looked at the proceedings much in the last decade. I am pretty sure it is one of those "every one who attends has a paper" conferences (which is fine with me). Back when I was doing research in the early 90's, I recall some decent papers from this event - not as high quality as some other conferences. With Dan Russell and Jonathan Grudin organizing this minitrack of papers, then I am sure these papers will be good.

I see lots of other things of interest to information architects. The mini-track on Using Information is part of the Digital media track with other minitracks like information retrieval and search effectiveness. There are also parts of other tracks on HCI, socialware, semantic web and many knowledge management topics. There will be lots to keep an IA engaged.

June 15, 2006 is the deadline to submit. I plan on submitting something myself, if I can get the OK to travel. Hawaii in January sounds too good to be true!

I participated in the panel "Setting the IA research agenda" at the IA Summit in Vancouver yesterday. Great talks by the panelists (of course), but more importantly, excellent participation from the audience and a good birds-of-a-feather at the end of the day. See my presentation, background information, and other notes.

In a few years, will this panel be seen as an important kick-off to an IA research agenda, or will it be seen as just another event where we talked about "IA research" but did nothing about it?

I participated in the "Setting the IA Research Agenda" panel on Saturday, March 25, 2006 at the IA Summit. Here is an overview of my points, with links to background information I compiled in prepration for the panel, as well as some of my notes. A copy of my presentation is attached below.

Fellow panelists: Don Turnbull (panel organizer), Peter Morville, Jamie Blustein.

What is Research?

• "Active, diligent, and systematic process of inquiry aimed at discovering, interpreting and revising facts" (from Wikipedia entry on Research)
• Also, a collection of information about a particular subject

• Basic research: to further knowledge for knowledge's sake
• Applied research: producing results that may be applied to real world situations ("Research phase" within our ordinary projects)

• Exploratory: to help define the problem
• Constructive: develop a solution to a problem
• Empirical: collect data to test hypotheses (see scientific method)

• Research methods (a sample): Action research, Case study, Observation, Intuition, Interview, Data analysis, Simulations & models, Controlled experiments

Examples of IA empirical research:

• Peter Morville's September 2004 review of the literature with sections on Information seeking behavior, Structure and organization, Navigation, Search
• Larson & Czerwinski, CHI 98, Implications of memory, structure and scent or information retrieval (PDF)
• Yee, Swearingen, Li, Hearst, CHI 2003, Faceted metadata for image search and browsing (PDF)
• Bernard, Developing Schemas for the Location of Common Web Objects

What is a Research Agenda?

Agenda

• A list or outline of things to be considered or done (e.g., meetings)
• An underlying often ideological plan or program (e.g., political)

Research agenda

• List of things you want to see researched
• A bunch of questions you want answered
• Structured list of issues, that when researched, will serve some greater goal

Examples of some of my IA work where I include research agenda questions:

• Breadcrumbs
• Faceted browsing
• Personalization

In November, 2004, Peter Van Dijck posted A research agenda for information architecture with follow-up comments from various mailing lists. His research agenda included: Cognitive science (categories, search terms), Business theory (process model), Social Science & Anthropology (information sharing). Donna Maurer later posted that Maybe we do need IA research with these agenda items: Genre, Cognition and categorization, Facets (mental model, UI elements), Re-finding information.

A sampling of existing research agendas that are related to information architecture:

• Hypermedia, semantic web: Hypermedia and the Semantic Web: A Research Agenda: Links vs. relationships, Embedded links, Time-based hypermedia, CSCW
• American Library Association: Research Agenda for Library Instruction and Information Literacy: Learners, Teaching, Organizational context, Assessment
• Gartner: Enterprise Architecture Research Agenda Set for 2005: Management and Governance, Trends and Directions, Methods and Tools, Architecture in Practice by Industry, Cultural and Organizational Issues
• ACM, Human-computer interaction: Strategic directions in HCI: Strategic themes (Universal access to info, Education/learning, E-commerce, End-user programming, Information Visualization, Computer-mediated communication), Technological trends (Ubiquitous computing, Hardware, Natural modalities, VR), Design & evaluation methods, Tools
• Digital libraries: Interoperability, Scaling, and the Digital Libraries Research Agenda, Research Agenda for the Intelligent Digital Library, Metadata for Digital Libraries: a Research Agenda
• Shneiderman: Universal usability: Technology variety (supporting a broad range of hardware, software, and network access), User diversity (accommodating users with different skills, knowledge, age, gender, disabilities, disabling conditions, literacy, culture, income), Gaps in user knowledge (bridging the gap between what users know and what they need to know)

How would these overlap with an IA research agenda? How were these agendas built? How do they relate to funding?

Research agendas will have a lot of overlap with curricula. For example, the HCI "inventory of topics" figure from the ACM SIGCHI CDG gives the same overview of the field that a research agenda would provide.

A small sampling of IA curricula:

• Kent State IAKM: Information architecture, Information use, Knowledge management
• Indiana University SLIS - MIS: IA & design, HCI & communication, Strategic infomation management and leadership, Information retrieval systems design
• University of Texas - Austin IA Proposed Curriculum (PDF)

The punch line: Treat the IA Research Agenda as an IA project

• A research agenda is a framework for classifying existing research and (more importantly) planning future research
• Research agendas have an IA
• Apply our IA methods to create an IA research agenda

My notes

My notes from the discussion during the panel session, the BOF later in the day, lunch conversations, etc.

• Don's slides (insert link here)
• Peer reviewing: nice-to-have for me (helps filter the research) but I am also interested in less polished research
• A journal is neccesary but not sufficient
• What are the boundaries of the field?
• Jamie's slides
• Hypertext's aims and goals match pretty well to IA's
• Tenure challenges are key, IA research needs to be recognized for tenure
• Peter's notes are on flickr
• Agenda must be bigger than findability
• Yes, we do need IA research
• Label it "IA research" - semantics matter
• Good sign: practitioners getting stalked by graduate students
• Basic vs. Applied research
• Closet research literature review in industry
• Good sources of IA research: ACM DL, JASIST
• Karl Fast's comments: Agenda is about asking the RIGHT questions, respecting other disciplines
• One goal for an agenda: $50mil grant from NSF
• Information literacy as a key agenda item (long term , hard to do)
• usability.gov and SURL as examples of making research usable by practitioners
• "Best practices" may be all that we need for the current set of problems
• Tools to help with IA research (e.g., a way to analyze web page layouts across a lot of sites)
• Directory of IA researchers: list of people who are doing research, updated overview of each project, easy to contact them
• Takes special skills to summarize research for a practitioner (researcher cannot do it)

Panel coverage

Blog entries and other discussions of this panel. If you know of others, please add them as comments - thanks!

• Abe Crystal's report on the Summit includes the panel. "I believe integrating and applying social and behavioral science results to IA problems could reinvigorate both IA practice and social science research."
• The confluence of research and practice in information architecture - Karl Fast's recap of research at the IA Summit.