Research Processes Across the Disciplines

The brief arrives as a claim about what the problem is. The first research act is to refuse that claim as given. Schön's distinction is the operative one: professionals do not merely solve problems presented to them, they set the problem — naming what will be attended to and framing the context in which it will be addressed.⁷ Rittel and Webber's account of wicked problems supplies the reason this matters: such problems cannot be definitively formulated, there is no stopping rule, and solutions are not true-or-false but good-or-bad. Crucially, the formulation of the problem is the solution — you cannot understand the problem without already committing to a way of resolving it.⁵

Failure modeAccepting the client's problem statement as the research question. The resulting project may be well-executed design, but it has not conducted inquiry — it has only answered a question someone else already settled.

Design does not proceed by analyzing a problem fully and then generating a solution. Cross and Dorst observed that designers develop problem-space and solution-space in parallel, with activity in each cross-fertilising the other: a tentative solution reveals a feature of the problem that was invisible before, which in turn reframes what would count as a solution.^3,6 This is the mechanism behind Cross's characterisation of designerly cognition as solution-focused rather than problem-focused — designers attack ill-defined problems by proposing solutions and learning from the proposal.³

Failure modeTreating iteration as mere refinement toward a fixed target. If the problem statement never changes across iterations, co-evolution has not occurred and the "research" is production.

Here the tradition splits along Frayling's axis, and the split is consequential for what the research can claim.¹ In research through design, the artifact is the instrument: it is built in order to find something out, and the finding is not reducible to a proposition stated in advance. Cross characterized designers as using codes — a two-way translation between abstract requirements and concrete objects, so that the properties of the material shape the process even as the process shapes the object.³ The prototype is not an illustration of a hypothesis; it is the site where the hypothesis is formed.

Failure modePost-hoc rationalisation — producing an artifact by ordinary practice, then narrating a research question it retroactively "answers." The tell is that no design decision could have come out otherwise.

This is where Frayling's argument becomes prescriptive rather than descriptive. He grants that design knowledge is characteristically tacit rather than propositional — but then argues the tacit element is not sufficient on its own, since if it were, every practicing artist would thereby merit a doctorate. His resolution is that practice must be complemented by documentation: a report that renders the tacit communicable, yielding an output combining internal and external knowledge.^1,2 Schön supplies the finer-grained mechanism, distinguishing reflection-in-action (the practitioner's thinking while doing, reshaping the work as it proceeds) from reflection-on-action (the retrospective account).⁷

Failure modeThe reflective account written as promotional narrative — a case study of a success, with the failures and reframings edited out. This destroys precisely the evidence that made it research.

The final stage is the one design research has least settled. A designed artifact is radically particular; research claims to produce knowledge that travels. Recent scholarship names this directly as the transferability problem arising from the Frayling typology.⁶ The proposed solutions do not agree with one another, which is itself worth teaching: some argue for intermediate-level knowledge (design patterns, strong concepts) pitched between the particular instance and general theory; others (Gaver) argue that research through design should not aspire to theory in the scientific sense at all, and that annotated portfolios preserve the specificity that abstraction destroys.^8,9

Failure modeOverclaiming generality from a single project, or its mirror — refusing any claim at all, leaving the work as an undocumented portfolio piece.

Artistic research rarely begins with a question and a method chosen to answer it; it begins inside a practice that is already underway. The first disciplined act is therefore a demarcation. Candy's distinction is the standard one: practitioners routinely "research" as part of everyday practice — searching out new understandings and techniques — but that searching is directed at the individual's goals of the moment. Practice-based research begins when the inquiry commits to adding to a shared store of knowledge, in a form others can access and contest.⁴⁴

Failure modeRelabeling ordinary studio searching as research. If nothing about the practice changes when it becomes "research" — no demarcated question, no documentation, no shared-knowledge commitment — the label is administrative, not epistemic.⁴⁴

The studio phase is where artistic research makes its distinctive epistemological claim: that materials and processes are not vehicles for executing a thought but participants in producing it. Barrett and Bolt's collection builds this position — drawing on Carter's notion of material thinking and Bolt's account of praxical, handling-based knowledge — into a methodological argument: what the hand and the material learn together cannot be fully anticipated by, or reduced to, a prior proposition.⁴¹ The philosophical underpinning is Polanyi's tacit dimension — we know more than we can tell — which Frayling had already imported into the art-and-design research debate.^1,48

Failure modeThe work as illustration — a thesis held in advance, executed in material form. The tell: no studio event ever surprised the researcher or altered the claim.

Haseman's manifesto names what is distinctive about this stage: performative research presents its findings in the symbolic forms of the practice itself — image, object, performance, sound — not solely in the discursive prose of the thesis. The claim is not reported by the work; it is enacted in it.⁴³ Borgdorff's parallel formulation is that the knowledge at stake is embodied in artworks and practices, and that unpacking it into propositions is always possible only partially and after the fact.^38,39 This is the strongest and most contested plank of the tradition — see the debates panel.

Failure modeThe work as accompaniment — an artifact exhibited alongside a written argument that does all the epistemic work. If the text could stand without the work, the research was not artistic research.

Frayling's argument returns here as a demand: the tacit element alone is insufficient, so practice must be complemented by an account that renders it communicable.¹ The tradition's instruments are the exegesis — the written component of the practice doctorate, whose proper relation to the work has been argued over since the degree existed^42,46 — and, more recently, the exposition: the Journal for Artistic Research's format, hosted on the Research Catalogue, in which text, image, audio, and documentation are composed as a single navigable object, an attempt to make the writing and the work one epistemic unit rather than a work plus its explanation.⁴⁵ Nelson's "complementary writing" stakes the moderate position: the writing does not translate the practice, it does different work — locating the practice in its lineages and making its knowledge claims examinable.⁴²

Failure modeThe exegesis as extended caption — describing the work, praising it, and narrating the artist's biography while making no examinable claim. Its mirror: the exegesis as independent essay, betraying that the work contributed nothing.

Artistic research answers to two audiences whose criteria do not reduce to one another: the artworld, which validates through exhibition, curation, and critical reception, and the academy, which validates through examination and peer review. The doctoral apparatus is where the collision is institutionalized — Nelson's protocols are largely an attempt to give examiners criteria that respect both⁴² — and the skeptical literature, collected around Elkins's examination of the studio-art PhD, questions whether the reconciliation is possible or even desirable.⁴⁷ The Journal for Artistic Research's peer-reviewed expositions are the field's most developed attempt to referee practice on its own terms.⁴⁵

Failure modeVenue substitution — treating gallery acceptance as scholarly validation, or demanding laboratory criteria that dissolve the art. Each error concedes the field's central claim: that the work can be rigorous as work. Sullivan's position is the direct rejoinder to the second error — borrowing the social sciences' methods is not what makes inquiry legitimate, and the visual arts can meet the standard of systematic inquiry on their own terms.⁴⁰

The move that defines the tradition is the deliberate suspension of the hypothesis. Glaser and Strauss developed the approach precisely by questioning whether a scientific method of verification was appropriate to what they were studying — dying patients and the awareness contexts around them.¹³ The researcher enters with sensitizing concepts (orienting notions that suggest where to look) rather than operational variables (specifications of what to measure). This is not a lack of rigour; it is a different location for rigour, which arrives later, in the discipline of comparison.

Failure modeA covert hypothesis. The researcher has a theory but does not declare it, then "discovers" it in the data — the inductive form with the deductive substance.

Sampling in this tradition is not fixed in advance and is not designed for representativeness. It is theoretical: cases are chosen for what they can reveal about the categories currently emerging, so the sample grows in response to the analysis. This makes collection and analysis simultaneous rather than sequential — an inversion of the empirical-scientific order that is easy to mistake for sloppiness if you carry quantitative expectations into the reading.

Failure modeConvenience sampling dressed as theoretical sampling — interviewing whoever was available and describing it post-hoc as theoretically motivated.

This is the tradition's signature procedure and the one most often misapplied. As data is collected it is continually compared against existing data to surface similarities, differences, and patterns; the process is iterative and runs throughout the study, from initial coding through theoretical sampling and memo-writing.^11,14 Glaser's original formulation proposed a hybrid model: exhaustively coding everything hinders the generation of new hypotheses, yet abandoning coding leaves the analyst merely inspecting data and writing memos. The resolution is to re-examine a code each time something is added to it, so theory is continually created or refined rather than fixed at the outset.¹¹

Failure modeThematic analysis relabelled as grounded theory. Coding data into themes and reporting frequencies is a legitimate method, but it is not constant comparison and does not produce a grounded theory.

Comparison continues until researchers reach theoretical saturation — the point at which no new information or insight emerges from further data.¹⁴ This is the tradition's answer to the question quantitative research settles with a power calculation, and it is genuinely different in kind: saturation is a property of the categories, not of the sample size. It is also the claim most frequently asserted and least frequently evidenced.

Failure mode"Saturation was reached" asserted with no supporting evidence — the single most common unfalsifiable claim in qualitative reporting.

A grounded theory must, in Glaser's terms, work and have relevance — it must explain the studied phenomenon analytically — and, like any theory, must change when conditions change.¹² Quality is described not as validity and reliability but as trustworthiness — credibility, transferability, dependability, confirmability³⁰ — assessed as concordance between data and result. Respondent validation functions as a form of triangulation. The disposition required has been characterized as "disciplined restraint" — reflexively questioning one's own interpretations rather than defending them.¹²

Failure modeImporting quantitative criteria wholesale — reporting inter-coder reliability as though it established truth, when what it establishes is agreement.

Before analysis begins, the researcher decides what counts as the object and what counts as its context — and that decision is not neutral. Choosing to read a poster as an isolated composition, as one item in a campaign, or as an instance of a national visual idiom produces three different objects, and forecloses different findings. In design scholarship this stage is routinely skipped, which is why so much visual analysis reads as assertion: the object was never argued for.

Failure modeReading a designed artifact from a low-resolution image online and treating that as the object, silently importing the reproduction's distortions into the analysis.

The discipline of humanities method is that description precedes interpretation and is held separate from it. Panofsky's three-level scheme formalizes this: pre-iconographic description (what is literally depicted — forms, lines, figures), iconographic analysis (what conventional subject or theme those forms carry), and iconological interpretation (what the work discloses about the underlying principles of its culture and moment).¹⁸ The order is not optional. A reading that leaps to iconology has no evidence to fall back on when challenged.

Failure modeInterpretation smuggled into description — writing "an aggressive red" rather than "red, saturated, occupying 40% of the field," and then treating the aggression as a finding.

The choice of analytic lens is a theoretical commitment, and in semiotics the two founding models are not interchangeable. Saussure's model is dyadic — signifier and signified — and holds that meaning arises from difference within a system of language; the relation between the two is arbitrary, and there is no third term for the object in the world.¹⁹ Peirce's model is triadic — sign, object, interpretant — and admits the referent, yielding the icon/index/symbol trichotomy in which resemblance and causal connection do real work that Saussure's system cannot accommodate.²⁰ A photograph is indexical for Peirce; Saussure has no equivalent category. Choosing arbitrarily between them, or blending them, is the most common error in design semiotics.

Failure mode"Semiotic analysis" as a decorative label — invoking signifier/signified while doing what is actually iconographic description, with no systemic account of difference.

The interpretation is the claim: this artifact means this, produces this effect, or does this ideological work. It is offered knowing that a rival reading is always possible; the standard is not proof but accounting — the better reading explains more of the object's features, and explains the features the rival cannot. Critical traditions add a further move: asking not only what the artifact means but whose interests that meaning serves, and what it naturalises as obvious.

Failure modeUnfalsifiable interpretation — a reading so accommodating that no feature of the artifact could have counted against it.

The final move is to locate the reading within an existing discourse — showing what it corrects, extends, or displaces. In the humanities the literature review is not a preliminary formality but part of the argument itself: a claim's significance is constituted by what it revises. Evidence is marshalled through citation, and the standard of scholarship is that a reader can follow every claim back to the object or the source.

Failure modeThe ahistorical reading — applying a contemporary framework to a historical artifact without acknowledging the anachronism, and mistaking the resulting friction for a finding.

The hypothesis must be stated before the data are seen, and it must specify what observation would count against it. This is the discipline's core commitment and the one most often violated in applied design research, where a study is run, an interesting pattern is noticed, and the hypothesis is written afterwards to fit it. Popper's point is that a theory which cannot fail cannot inform: the more a hypothesis forbids, the more it says.²³

Failure modeHARKing — hypothesising after results are known. Formally indistinguishable from confirmatory science in the write-up, and epistemically worthless.

The theoretical construct ("legibility," "trust," "cognitive load") must be turned into something measurable, and the gap between construct and measure is where design research most often breaks. Campbell and Stanley's framework remains the standard apparatus: distinguishing internal validity (did the manipulation cause the effect?) from external validity (does it hold beyond this setting?), and cataloging the threats to each — history, maturation, selection, instrumentation, regression to the mean.²⁴ A quasi-experimental design, lacking random assignment, is not thereby invalid, but the researcher must argue explicitly which threats remain live.

Failure modeConstruct slippage — measuring reading speed and reporting a finding about "legibility," or measuring task completion and reporting a finding about "delight."

The value of the design is realised only if the protocol is executed identically across conditions. In design contexts this is harder than it sounds: the experimenter often is the designer, and knows which condition is the one they hope will win. Blinding — of participants where possible, of the person administering and scoring where at all feasible — is the standard defense against the researcher's own expectations leaking into the measurement.

Failure modeThe designer running the test on their own design, unblinded, and recording "observations" that reliably favor it.

Analysis follows the plan set before the data were seen. The contemporary standard has shifted decisively away from reporting a p-value alone: a statistically significant result may be practically trivial, and a non-significant result is not evidence of no effect. What is reported is the estimated effect size with its confidence interval — the magnitude of the difference and the precision with which it is known.

Failure modep-hacking — testing many comparisons and reporting the significant ones; and the inverse error, reading a null result as proof of equivalence.

The result becomes knowledge only when others can attempt to reproduce it. This is the stage where design research has been weakest — proprietary usability findings, unreported stimuli, unavailable materials. Transparent reporting of the stimuli, protocol, data, and analysis code is the condition of the tradition's claim to authority; without it the falsifiability commitment at stage 01 is empty, since nobody else is in a position to falsify anything.

Failure modeThe unreplicable design finding, circulated as a rule of thumb ("users don't scroll," "the fold matters") long after its original conditions have vanished.