How Can We Know What Works?

A Project Report on the Value of Photographs
Author
Translator
Pages: 2-10
Year: 1987
Notes on S.I

Notes on Structural Integration – March 1987 – 87/1

Volume: 87/1

Once after a first session of Rolfing I had forgotten to change the numbers from zero to one before taking pictures. When I then looked at the developed pictures I was shocked to find out that I had great difficulties to decide which were the before and which the after pictures. And afterwards I wasn’t at all very certain about my decision. Shocked I was because before this incident I had always seen quite a lot of changes in the photographs when I knew them to be pictures taken after the session. Now I was not so certain anymore whether I had really seen those changes or whether I had made myself believe to see them just because knowing they were after pictures made me know what to see.

<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-1.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-2.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-3.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-4.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-5.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-6.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-7.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-8.jpg’>
Series 1

Since I am vain enough to think that what happened to me can happen to most others, I suggested to other Rolfers to get a little project started on the value of photographs. I wanted to find out: Do photographs show Rolfing changes in such a way that Rolfers can identify them and distinguish them from changes that have other causes? Please note that in this question I do not doubt that Rolfing causes changes or that Rolfers can see changes. I merely question whether photographs are a means by which Rolfers can identify the effects of their work and distinguish these from the effects of other factors like simple relaxation, accidental changes in position, or suchlike things.

I consider this question to be of considerable importance. For, if it was answered with yes, then we would be able to use photographs to find out for example what kind of hamstring-work produces best results with a certain structural type. It would be an answer to the question: How can we know what works? Without such an answer we are left to subjective claims that cannot be proved or refuted, and therefore we are left to the contentions of those who most persuasively can seduce or bully others into believing that they do ?great work?.

Why use photographs when you can look at the client himself? The client is three-dimensional, you can see him move and breathe, and you can see him change. That is the best test whether a Rolfing intervention works. This very legitimate and widespread view of the problem for me personally has the drawback that rarely – if ever – do I see the change happening, and equally rarely – if ever – do I have an exact enough memory of how things looked and moved before the intervention to precisely judge the change affected. But there are also some more objective drawbacks: the change has to be documented in some way by as exact a description of the before and after situation as possible which implies measurements. These tend to be tedious both for client and practitioner and do not permit for recognizing change in the whole ?Gestalt?. And naturally a majority vote of those who look at a client and agree that ?something happened in the back? will not do at all, since many psychological experiments have shown how such protocols are open to influences like pressure by authoritative persons or wanting to conform with a belief system to the point where things are ?seen? no independent viewer can discern.

<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-9.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-10.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-11.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-12.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-13.jpg’>
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-14.jpg’>
Series 2

Result no.1: Rolfers can distinguish the photographs of persons who had a first session of Rolfing from those who had Massage, Reiki, or Nothing in a significantly more than probable frequency (2,5% novae).

Result no.2: Rolfers can best distinguish post-Rolfing photographs from photographs of persons who have had Nothing (0,5% novae).

Result no.3: Without the photographs after Nothing Rolfers tend to be able to distinguish post-Rolfing photos from those after Reiki or Massage, but in 30 of 100 experiments the same result could have been produced by sheer chance (30% novae).

Result no.4: Rolfers particularly fail to be able to distinguish post-Reiki pictures from post-Rolfing pictures. Nearly as many post-Reiki pictures are taken to be post-Rolfing pictures as post-Rolfing pictures are correctly identified (0,05 novae).

Photographs have a bit more of an objective quality. You can look at them over and over again, compare the before and after situation, compare it to what viewers say about it, and even measure whether what they say is true. Photographs can be reproduced in any number so that they can be scrutinized in seclusion without group influence or other pressures. And although one does lose movement and the third dimension, still a lot of the ?Gestalt,? changes can be represented in a photograph. Therefore, I think photographs could be the ideal instrument for this journal to practically test the theories on Structural Integration and the technique of Rolfing. This should be enough to demonstrate the relevance of such a study on the value of photographs for the development of practical research.

Table I

<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-15.jpg’>

<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-16.jpg’>

<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-17.jpg’>

<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-18.jpg’>

(AR or A: Advanced Rolfer; ER: Experienced Rolfer; BR or B: Beginner; T: Teacher; horizontal numbers: photograph codes)

Let me introduce to you this little project by asking you to put yourself to the same test the participants had to go through, only on a much smaller scale: Series 1 shows four sets of photographs of four people marked with a code number and with a zero for before and a one for after. One of these persons has received a first session of Rolfing without ever having been Rolfed before. One of them received a full body Swedish Massage lasting about one hour. Another person of the four received a one hour session of Reiki. Reiki claims to be a healing treatment by Universal Energy which practitioners have to be initiated to by a mystic ritual. Apart from these esoteric assertions, its simple physical reality is like this: in a session of Reiki the client lies on the table and the practitioner softly places his palms for several minutes at a time on all parts of the client’s body. So without its metaphysics it is one hour of meditative body-awareness and relaxation under the caring attention and touch of another person. Finally, one of the four persons got a session of Nothing, i.e. after the pictures were marked ?0? the person was asked to walk around the table and then the after pictures – marked ?1? – were taken. Sometimes the model’s hair was rumpled and the underwear displaced to create the outside marks of a Rolfing session.

Table II

Reiki
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-19.jpg’>

Massage
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-20.jpg’>

Nothing
<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-21.jpg’>

(Horizontal numbers: photo code; vertical numbers: viewers code; c: correct; w: wrong; r: rolfed)

Chi-Square for Swedish Massage: 16 = 0,1% (highly significant)

Chi-Square for Reiki: 2,77 = 10% (not significant – just a trend)

Chi-Square for Nothing: 36 = 0,01% (definitely significant) Chi-Square for the whole: 45,37 = 0,001% (definitely significant)

Result no.1: Rolfers can in general recognize the photographic changes of a Rolfing session when confronted with the alternative between the photographs of a Rolfing session and a session of another method to the same person (significant).

Result no.2: Surprisingly enough Rolfers are least able to significantly distinguish between Rolfing session photographs and those taken after a session of Reiki. Whether this is due to the relaxating effect of Reiki, the caring touch, or the occult qualities cannot be deduced from statistics.

Result no.3: Rolfers can distinguish best when the alternative is between Rolfing pictures and those after no treatment.

Great care was taken to exclude any artificial variances because of faulty picture-taking. The camera was fixed at exactly the same spot and angle. The client’s toe-nail of his or her big toe always was on the same spot so that the client always was in the same relative position to the camera. The clients always were instructed ?to stand easy without any conscious pose?. To make comparison easier a square-centimeter grid was used as a background.

Now look at one set of pictures after the next and decide whether the changes that you see – if you see any – between ?0? and ?1? look like the sort of changes that could be caused by the moment to moment fleeting changes in breathing and swaying of living organisms, but no more. Then probably you have a treatment of Nothing. Or do the changes look like those that might happen after an hour of relaxing, body-centered, soft and caring touching? Then your bet will be on Reiki. Or do they look like the kind of changes that are induced by one hour of vigorous massage to the soft tissue without special attention to the whole person? Then it might be Swedish Massage. Or do they look like the changes that you expect after a first session of Rolfing? Can you describe these expected changes? Is it: more length in front, separation of pelvis and thorax, lengthening in the hamstrings, more horizontality in the pelvis, and a more harmonius pattern of breathing? These were the goals for the first session as taught to me in my basic training. If your definition of goals IS different, we already have one important possible source why we might end up ?recognizing? totally different pictures as being representative of Rolfing changes.

I urge you to really take the time and come to a decision on which set of pictures m your opinion is representative of which method. You can add or subtract weight to your decision by giving it a number between 1 and 6 for its degree of certainty (1 = wild guess, 2= uncertain, 3= fairly certain, 4= certain, 5= very certain, 6= dead sure). What number do you give to your decisions? Please write your decision on a piece of paper before you continue in this text, otherwise you will miss the whole difficulty of such a decision and you will fail to understand the results of the project.

If you really did put yourself to the test, you will now be in the state of confusion and frustration that caused me to start this project. Imagine the state eight of your colleagues were in after they had to take such decisions on 36 sets of photographs: 9 after Rolfing, 9 after Reiki, 9 after Massage, and 9 after Nothing. Just to enable you to reconsider your own decision, let me give you the answers of your colleagues on these four, before I present to you the correct answers.

No. 35: Rolfing 6, Nothing 2, Nothing 4, Rolfing 4, Rolfing, Rolfing 2, Rolfing 5, Massage 2. The two Nothings came from two eminent Teachers of Rolfing, so don’t be swayed too easily by the many Rolfings.

No. 51: Rolfing 5, Rolfing 6, Rolfing 5, blank, Massage 3, Massage or Reiki, Nothing 2.

No. 71: Nothing 3, Rolfing 3, Nothing 4, Rolfing 3, blank, Massage 3, Rolfing 5, Massage 2.

No. 54: Massage 2, Reiki 3, Nothing 4, Reiki 1, Rolfing, Nothing 3, Reiki or Massage, Reiki 1.

Naturally I knew that if the results of the project were a disaster, as I expected them to be, many Rolfers would try to evade the issue by saying: ?Probably the Rolfer who gave the session was no good, or those who looked at the pictures were not experienced enough?. Therefore I asked three different Rolfers to do the Rolfing sessions. The first three were given by an Advanced Rolfer, the second three by an Experienced Rolfer, and the third three by a Beginner. The same principle was applied in the selection of those who functioned as judges: 3 Teachers, 3 Advanced Rolfers, and 2 Beginners.

Table I lists the results in detail. Now you can see how you scored.

If you missed, here is a second chance: Look at the pictures of Series 2. This time the set-up obviously is different. You do not have to decide which of different persons got what, admittedly a very difficult thing to do. Now you see three pictures, one marked ?0? for before, one marked Bait and one marked ?bit. One of the two pictures marked ?all or ?bit was taken after a session of Reiki, Massage, or Nothing; the other after an additional first session of Rolfing. It is your job to decide which of the two was taken after one of the first three and which after Rolfing. Again you can give your decision a number between 1 and 6 for its degree of certainty. Do this now, please.

Probably you found this still difficult but much easier than the first test. I devised it because I was quite sure that everybody would fail on the first test. So I decided to treat all the 27 persons who had gotten Reiki, Swedish Massage, or Nothing in the first test with a free first session of Rolfing afterwards. They were Rolfed by two Beginners, the pictures were taken and arranged as ?a? or ?bit by the flip of a coin. These 27 sets of triple pictures were then sent out to about 20 European Rolfers of whom 8 responded in the course of three months. Here again – for suspense – compare your results with those of the 8 European Rolfers:

No. 13: a3, b6, b, b3, a3, a5, b4, b3.

No. 45: a3, a6, a, b2, a3, a4, a5, b4.

No. 61: a3, a4, b, a4, a4, a3, a5, a2.

Now – after you have reconsidered your decision – check for the results in Table II.

Table I shows the results of the first test. To find out whether you were right in your decision for example on number 35, you have to look for that number in the first row. Below it you then find what the eight judges thought that number 35 received. You can see that one judge only marked those he or she decided were Rolfed, another refused to differentiate between C (Massage) and B (Reiki). On the left border you can see whether the judges were Teachers, Advanced Rolfers, or Beginners. And in the block for method A (Rolfing) you can see in the top row whether the persons were Rolfed by an Advanced Rolfer, an Experienced Rolfer, or a Beginner.

Table II shows in the top rows whether ?all or ?bit was correct for Rolfing and what method preceded the Rolfing. Below each code number the decisions of the viewers are listed. The bottom line lists how man decisions were wrong for each code number. On the right margin the numbers of correct and wrong decisions are listed for each viewer.

Interpretation of Results

First of all I dare say that my doubts were well justified. In Table I it can be seen that nine times pictures of persons who had only walked around a table were taken to be pictures of persons after a first Rolfing session. Table II shows even worse results: in the Nothing block where of the three pictures one showed the result of a first session of Rolfing and the other showed the result of a trip around the table in the very same person, 18 times Rolfers take the picture after the walk to be a representation of the changes of a Rolfing session! That is a lot and should warn us to be very careful in our interpretation of pictures. We can easily be fooled.

On the other hand I must admit that my worst expectations have not come true. They were that throughout all methods Rolfers would only ?recognize? the correct pictures by sheer chance. This worst expectation is represented in Graph 1 by the horizontal dotted line (one quarter in every method: 72_4=18). But obviously the A-line, i.e. in each method the number of picture sets that were correctly (in A) or incorrectly (in B,C,D) identified as post-Rolfing pictures, does not follow that line of worst expectations. The same is true for the D-line, i.e. in each method the number of picture sets that were thought to be taken after a walk around the table (in A,B,C incorrectly, in D correctly). When A is highest, D is lowest and the other way around. Since statistically there is only the chance of 1 out of 1000 that this is caused by chance only, it can be seen as proven: Rolfers – not always but generally – can distinguish the pictures after a first session of Rolfing from those after a walk around the table. Hurrah for that! – it is something we can build on.

The courses of the A-line and the D-line in Graph 1, however, suggest more than that. They show such a nice and steady descent from the maximum that they seem to imply: Rolfers can also distinguish pictures taken after Massage or Reiki from those taken after Rolfing and after Nothing. But as the dotted B- and C-lines show by their parallel course to the line of worst expectations, this is murky ground (only a 30% significance niveau, i.e. 30 out of 100 experiments would show the same result by chance only). Statistically it is only by the addition of the post-Nothing pictures that Rolfers show a significant (2,5% niveau) ability to distinguish Rolfing pictures from those of other methods. Without the Nothing sessions as an easy contrast, we look bad.

<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-22.jpg’>

But Graph 1 holds another real surprise: Normally I would have thought that Swedish Massage is much more like Rolfing than Reiki and therefore I expected that post-Massage pictures would be more often taken to be post-Rolfing ones than pictures taken after Reiki, which knows no manipulation of tissue but only soft outside touches. But the opposite is true: Post-Reiki pictures are so often identified as post-Rolfing pictures that there is no statistically significant difference between the two, while post-Massage pictures are only identified as post-Rolfing pictures on the line of average expectancy. And strangely enough, pictures after Swedish Massage are more often taken to represent a treatment of Nothing than post-Reiki pictures, although compared to Swedish Massage Reiki seems much more like nothing.

<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-23.jpg’>

Looking at the line of correct answers in Graph 2, you can see the other side of the same puzzling story: Rolfers have most difficulty in identifying correctly the post-Reiki pictures. Again I would have expected the post-Massage pictures to cause the greatest difficulties and to be below the expected line of equal distribution (lower than in Graph 1 because of less responses to B,C, and D).

<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-24.jpg’>

Graph 3 shows the results of Table II: The Mistake Line shows how often Rolfers were wrong when they had to decide whether ?a,, or ?bit were the pictures of a person taken after a Rolfing session. As I expected, the results are much more encouraging than in Table I. The statistical significance in the ability to distinguish post-Rolfing pictures from pictures after the other three methods is much higher (0,001%).

But again it is clearly the contrasting post-Nothing pictures which make the difference (0,01%). Post-Massage pictures still are significantly (0,1%) distinguished, but the post-Reiki pictures are too close to the expected line of random distribution to be statistically significant (10%).

This naturally underlines the surprising result of Table I: It is hardest for Rolfers to distinguish pictures taken after a session of Reiki from pictures taken after a first session of Rolfing! This is true not only when different persons are looked at who got the different treatments but also when the two treatments are given to one and the same person, first Reiki, then Rolfing.

It is selfevident that statistics cannot explain this surprising result. Since we have no information on what the judges were looking for in the pictures, we can only speculate what caused them to see so frequently the changes they expect in post-Rolfing pictures in photographs of persons who had Reiki instead. Clearly it would be wrong to conclude from this that Reiki and Rolfing produce similar changes. The statistics only indicate that Rolfers cannot clearly distinguish these changes on photographs. This may be due to an insufficient clarity in the definition of changes expected from a first session of Rolfing. If such a definition is as vague as ?creating lifter, ?a better flow of energy?, ?more liveliness?, or other categories like this without a detailed description of the expected structural changes, then indeed it is not surprising that the effects of the two seem alike. Probably any good centering meditation would create the same effects, or any other method with such caring bodily attention common to both Rolfing and Reiki. It is the structural change that makes Rolfing special and if these specific structural changes are not in the center of the Rolfing sessions and of the observation of the judges alike, the results of a Rolfing session may understandably look much like those of a Reiki session.

This has to be tested in a follow-up study. Until then the practitioners of Reiki will use this study to claim that their work creates the same results as Rolfing by the force of the Universal Life Energy or some other wondrous miracle. Others may claim that the main effect of Rolfing is caused by the caring attention and less by the direct tissue manipulation – a legitimate speculation on the basis of the statistical results of this study.

But already now there are strong indications that the clear definition of structural changes is the crux of the matter. Naturally I was not able to remember in the second test which of the two, ?at? or ?bit, had been the Rolfing sessions. Yet with the clear definitions of the goals of the first session in mind and knowing that the practitioners who gave the 27 sessions shared these goals I was able to correctly identify 25 of the 27 sessions. And even better: when I gave the ?a,, and ah? pictures to a non-Rolfer and told her to look for more length in front and in the hamstrings and a better separation of pelvis and thorax (the goals that the two practitioners who gave the sessions had worked for), she quickly and correctly identified 24 of the 27 sets of pictures. That is by far better than any of the Rolfers did who participated in the project!

<img src=’https://novo.pedroprado.com.br/imgs/1987/1020-25.jpg’>

Graph 4 gives some support to this interpretation. It shows the average of correct answers for Teachers, Advanced, and Beginning Rolfers in Table I. An average had to be taken because some responded only to some of the methods. I have shown before that in the areas of Reiki and Massage guessing prevails and the results lie in the area of random distribution. In this area (B and C) of guessing Teachers score low. In the structurally decisive areas of Rolfing and Nothing Teachers score above Advanced Rolfers and these above Beginners. For me this is an important result, for it encourages my hopes that analyzing photographs is something that can be learned by experience and exercise.

Seen from this perspective, the project results could be interpreted as demonstrating the lack of definition and consensus of what the expected changes of a first session of Rolfing should be. For, evidently one can only recognize something that one already consciously knows and looks for.

Seen this way, the results of this project reflect much less on the question whether photographs can be an instrument for judging what works. The important result is to realize the lack of definition and consensus on what changes should be looked for. It is the murkiness of our goals that makes it possible for the effects of a first session of Rolfing to be mixed up with the effects of something like Reiki, which is so fundamentally different from Rolfing in all respects except ?good intentions,,.

Therefore, in conclusion, the question: ?How can we know what works?? – finds a surprising answer: We first have to define and describe very clearly and in detail how that which we want to change should look afterwards, then and only then photographs may be a useful tool to identify whether these changes have been brought about or not. Thus this project underlines the necessity of the work the authors of this journal have undertaken to do.

Postscriptum

There were some other less important results which are more of a curiosity than of central relevance. One is the widespread expectation that sessions you give away for free show not as good results as those that are paid for. The statistics of this project point to the opposite: of the paid sessions only 10 out of 40 possible identifications were made, while of the free sessions 16 out of 32 correct identifications were made. But this has to do with the other curious fact: I expected most correct identifications for the sessions the Advanced Rolfer gave, then for those of the Experienced Rolfer, and finally for the Beginner. The opposite turned out to be the case. Each had 24 possible identifications. Of these the Beginner got 14, the Experienced Rolfer 9, and the Advanced Rolfer 3. Those who want to conclude from this on the quality of these Rolfers I want to remind that the number of correct identifications probably shows more about a consensus on the goals of a session then on the quality of the work or the extent of changes. Only after such a consensus is established among those who give the sessions and those who judge the pictures can the numbers of correct identifications reflect on the success or failure of sessions.

Finally, I want to express my thanks to all those who participated in this little project: 36 volunteers, A. Boeckh for the following Rolfers for doing the sessions or being judges: Cordelia Alber-Klein, Manuela Brinkmann-Hartmann, Harvey Burns, Lynne Christianson, Hans Diepold, Hans Flury, Heidrun Gelenk-Borst, Oskar Holzberg, Seamus Keane, Peter Melchior, Tom Myers, Luigi Negro, Stanley Rosenberg, Michael Salveson, Robert Schleip, Beverly Silverman, Jan Sultan, Bernadette van Boxel, Thomas Walser.

To have full access to the content of this article you need to be registered on the site. Sign up or Register. 

Log In