FW: [governance] [Fwd: Outside evaluation]

Wed Aug 13 10:47:29 EDT 2008

Since only a couple of people seem to have read this below when I first sent
it out I'm taking the liberty of resending it...

I don't agree with those who are indicating that an "evaulation" is the
right way to go here... In fact given what I know about evaluation and
evaluation contracting in the UN system I would drop the terminology of
"evaluation" altogether since once it is termed that way and given over to
the UN to administer there are a range of criteria, standards, contracting
procedures, and operational processes that move it out of anyone (outside
the system)'s direct control or capacity to influence.

I would rather use the terminology of "consultative assessment (or review)
with independent support" or some such.

Please see my original note copied below for a more extensive argument
concerning this...

MG 

---------------------

-----Original Message-----
From: Michael Gurstein [mailto:gurstein at gmail.com] 
Sent: August 7, 2008 2:11 PM
To: 'governance at lists.cpsr.org'; 'Jeanette Hofmann'
Subject: RE: [governance] [Fwd: Outside evaluation]

Based on fairly extensive experience with "evaluation" over many years
including in the context of the UN system I would observe that as is often
the case there is an attempt here to shift what is essentially a "political"
decision (continuation of the IGF) into the technical sphere (i.e.
evaluation as the "objective determination of the achievement of a
pre-defined set of (program) goals/objectives").

The problem with doing this is that it generally puts the "political" onus
on the evaluator/evaluation and thus on the process of determination of
evaluation criteria and the selection/personality/biases of the evaluator.

A tough position for the evaluator for sure but also very tricky in terms of
ensuring that one's (e.g. IGC's) specific interests are adequately
represented in the process and the outcome.

On the other hand it is often very useful to have a qualified third party
draw up the logic models, put the relevant information together, interview
the various stakeholders and so on.

Another way of proceeding, and one perhaps more in keeping with the spirit
of the IGF and multistakeholderism would be that the evaluation be seen as a
multistakeholder "process" rather than the "product"/output of a specific
evaluator. 

The "evaluator" would thus be chosen to inform and support the evaluation
process with the outcome of this process being a recommendation from the
participants in the process to the SG rather than as being simply the
outcome of the evaluator's internal analyses.

MG 

-----Original Message-----
From: Jeanette Hofmann [mailto:jeanette at wzb.eu] 
Sent: August 6, 2008 12:52 AM
To: Governance
Subject: [governance] [Fwd: Outside evaluation]

Hi everyone,

the MAG has recently discussed potential approaches to the upcoming 
evaluation of the forum. As has been mentioned on this list, there is 
also the option of an outside evaluation. Some MAG members like this 
idea, others expressed reservations.

In the following message to the MAG, Markus outlines the state of 
things, his interpretation of the Tunis Agenda, and the time table for 
the evaluation of the IGF.
jeanette

Dear colleagues,

In light of the feed-back received we will refrain from posting the draft
TOR for an external evaluation and asking for comments. Instead, we will put
the review process in general terms on the agenda for the September
consultations. The external review will be a sub-item under this agenda
item.

At this stage, it is useful to start the discussion on the review process.
Let me add a few thoughts on this issues.

While the Tunis Agenda is fairly precise in this matter, there is
nevertheless room for interpretation and a need for discussing how the
paragraphs related to the review need can  to be translated into action.

A few elements are given:

The actors:

-The Secretary General (he is charged with examining "the desirability of
the continuation of the Forum")

- Forum participants (they are to be consulted)

- UN Membership: the Member States are to take a decision, based on
recommendations by the Secretary-General

The timing: para 76 states: "within five years of its creation". Presumably,
this means the date when the Secretary-General convened the first meeting of
the Forum, in July 2006. The review therefore needs to take place no later
than July 2011. However, the Tunis Agenda allows for the review to take
place earlier. A review process should leave the door open and should not
prejudge a decision in either way. Should it be delayed until the last
moment, it would have a negative impact on the IGF. A decision in this
regard should be taken by the end of 2010 at the latest, should there be a
general desire for a 2011 meeting.

The UN Membership takes decisions in the framework of the annual General
Assembly which meets each year between September and December. Decisions by
the GA need to be prepared by subsidiary
bodies: in this case the CSTD and ECOSOC. For the GA to be able to take a
decision by the end of 2010, the Secretary-General will need to submit his
recommendations as part of his annual report on the WSIS Follow-up to the
CSTD. This report is prepared early each year. The elements for the
Secretary-General's recommendations therefore need to be ready by late 2009.

Para 76 mentions that there is a need for "formal consultations with Forum
participants". Does this relate to the annual meeting or to the regular
consultations in Geneva? It would be safe to assume that this para relates
to the participants at the Forum itself, as this is a far broader community
than the 'IGF insiders' who attend the Geneva meetings. Should this be the
general reading of this para, then the 2009 meeting in Egypt will need to
include a slot in the programme for these "formal consultations with Forum
participants."

This is my reading of the situation. Any other views are welcome. I will
also ask the legal services of the UN to give us their interpretation.

The Tunis Agenda leaves open how the "formal consultations with Forum
participants" should be prepared and carried out. We thought an external
input into this process in form of an evaluation could be helpful. In any
case, we should start "examining the desirability of the continuation of the
Forum" at the open consultations in May 2009 at the latest.

Best regards
Markus

____________________________________________________________
You received this message as a subscriber on the list:
     governance at lists.cpsr.org
To be removed from the list, send any message to:
     governance-unsubscribe at lists.cpsr.org

For all list information and functions, see:
     http://lists.cpsr.org/lists/info/governance

____________________________________________________________
You received this message as a subscriber on the list:
     governance at lists.cpsr.org
To be removed from the list, send any message to:
     governance-unsubscribe at lists.cpsr.org

For all list information and functions, see:
     http://lists.cpsr.org/lists/info/governance