[governance] IGF 2019 Workshop Evaluation process

Wed May 8 10:09:06 EDT 2019

Thanks for asking Arsene

I wrote a blog last year where I enumerated the problems with the
evaluation process. I copy paste the relevant section here.

*MAG and its curious approach to workshop selection*
Coming up with conference programs in a highly politicized environment such
as the United Nations is not an easy task, especially for an event like the
IGF where inclusion and diversity are fundamental and rightly important.
Naturally, there were always problems with shaping the program, deciding on
which sessions be included and what topics to be discussed. There was
always a tendency to give some space to sessions that don’t really engage
with Internet governance. But the IGF agenda is now at a stage of crisis.
In the early days of the IGF when the globally inclusive multistakeholder
concept was still largely unique, and it was gaining popularity, and when
people could still remember why they convened IGF, there was some kind of
order in the chaos of selection. IGF MAG could accept all of the proposals,
merge a few and let the participants decide on the sessions they wanted to
attend. MAG in those early years would not dictate the agenda and what
should be discussed to the extent that it does now.
But over the time, there was a change. Due to lack of resources and an
increase in the number of session proposals, MAG’s role became more
prominent in selecting the sessions and in shaping the agenda. That is when
it started to get more complicated. Perhaps it didn’t help that IGF works
under the auspices of the UN Department of Economic and Social Affairs
(UNDESA), and the MAG members are appointed more or less by UNDESA. Within
the UN, UNDESA is the agency that “holds up the development pillar of the
UN.” Is UNDESA influencing the IGF?
This year, to be more inclusive, MAG asked the public what it wanted to be
discussed at IGF. The response is summarized in this document.
<https://www.intgovforum.org/multilingual/index.php?q=filedepot_download/6701/1279>
According to the summary, twenty-two percent of the respondents wanted to
talk about cybersecurity, trust, and privacy; only seventeen percent
submitted digital inclusion and accessibility. And the rest was about other
topics. The results of the MAG process show a marked bias. Twenty-two
percent of the proposals received by MAG were about cybersecurity,  but
only twelve percent ended up in the top forty ranked proposals. In
contrast, only thirteen percent of the proposals received by MAG were about
digital inclusion and accessibility, yet thirty-two percent of the top
forty ranked workshops were about digital inclusion. This was highlighted
by *IGF Secretariat during MAG deliberations:
<https://www.intgovforum.org/multilingual/content/igf-2018-second-open-consultations-and-mag-meeting-day-1>*

“But then if you look at the top 40 proposals, there is some deviation from
what the most popular themes were in those …two other pools of proposals.
We have as the most represented theme digital inclusion when in the overall
proposal pool and the call for issues it was clearly cybersecurity. And
then following from that technical and operational topics, emerging
technologies, cybersecurity, development, innovation and economic issues,
evolution of Internet governance, and then human rights as last.”

How did this happen? This year the MAG’s working group on workshop
selection decided to assign each MAG member to a theme that the member had
expertise in and was more comfortable with. And they could reduce the
number of evaluated workshops by each MAG member because apparently, it’s
too difficult for one person to evaluate so many proposals. Some of the
deliberations and questions that have been raised about this issue among
MAG members *can be found at this archive list.
<http://intgovforum.org/pipermail/wgwseval_intgovforum.org>*

The problem with this method of evaluation is apparent. It prevents MAG
members from various stakeholder groups to assess the proposals
objectively. Even though the secretariat assigned an equal number of
members to each theme (and presumably considered stakeholder balance)
<https://www.intgovforum.org/multilingual/content/2018-igf-mag-virtual-meeting-vii-0>,
some proposals did not have more than three evaluators on the day of the
evaluation deadline. The Secretariat asked the MAG to do more evaluation
since 13 proposals had only 3 evaluations
<https://www.intgovforum.org/multilingual/content/2018-igf-mag-virtual-meeting-viii-0>
.

https://www.internetgovernance.org/2018/10/29/igf-embracing-inclusion-excluding-internet-governance/
Farzaneh

On Wed, May 8, 2019 at 9:28 AM sivasubramanian muthusamy <
governance at lists.riseup.net> wrote:

>
>
> On Wed, May 8, 2019, 6:42 PM Arsène Tungali <governance at lists.riseup.net>
> wrote:
>
>> Hi there,
>>
>> Just wanted to throw this outta here.
>>
>> At the MAG, we are evaluating workshop proposals (the initial phase)
>> and this goes until  May 12th. This is a tough job!
>>
>> I remember last year, there were a lot of unhappy people based on the
>> results of the evaluation. What did the MAG do wrong last year?
>>
>> Can you share what were some of the frustrations that came out of the
>> process last year? I know it might be late (or maybe not?) but if we
>> can quickly discuss here, I am happy to consider some of your inputs
>> and share to fellow MAG members.
>>
>
> Marginal errors in workshop evaluation
> are unavoidable. But the MAG could ensure that there is no orchestrated
> bias in the process towards or against a certain school of thought.
>
> Sivasubramanian M
>
>>
>> Regards,
>> Arsene
>>
>> ----------------------
>> **Arsène Tungali* <http://about.me/ArseneTungali>*
>> Co-Founder & Executive Director, *Rudi international
>> <http://www.rudiinternational.org>*,
>> CEO,* Smart Services Sarl <https://www.smart-kitoko.com/>*,
>> Tel: +243 993810967 (DRC)
>> GPG: 523644A0
>>
>> 2015 Mandela Washington Fellow
>> <
>>
>> http://tungali.blogspot.com/2015/06/selected-for-2015-mandela-washington.html
>> >
>>
>> (YALI) - ICANN GNSO Council Member
>> <https://gnso.icann.org/en/about/gnso-council.htm> Member. UN IGF MAG
>> <https://www.un.org/press/en/2018/pi2247.doc.htm> Member
>> ---
>> To unsubscribe: <mailto:igc-unsubscribe at lists.riseup.net>
>> List help: <https://riseup.net/lists>
>>
> ---
> To unsubscribe: <mailto:igc-unsubscribe at lists.riseup.net>
> List help: <https://riseup.net/lists>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.igcaucus.org/pipermail/governance/attachments/20190508/2a25ce76/attachment.htm>