Peer Review Board Selection Criteria
The questions below are taken directly from the draft criteria pages linked on http://wiki.openitp.org/peerreviewboard:start
Initial Filtering
| Criterion | Our Answer
|
| Is it open source? | Yes
|
| Is it relevant to our field? | Should be (see below)
|
| Is it a circumvention tool? | Yes
|
| Is it a host security tool? | No?
|
| Is it a secure communications tool? | Yes
|
| Is it aimed at a high-risk population? | Yes?
|
| Is it an evidence gathering tool for human rights? | No
|
| Is it a disaster response or democracy facilitation tool? | No
|
| Is it an end-user or middle-tier tool? | End-user(?)
|
| Do we have a coverage gap that this tool fits into? | ?
|
| Is this tool real or just vaporware? | Real
|
| Is this tool shipping? | Yes
|
| Is this tool in production? | Yes
|
| Does this tool have actual users? | Yes
|
| Does this tool fill a currently unmet need? | Yes - a decentralised low-latency anonymous network
|
| Does this have unique advantages such as usability or localization? | Yes? - Built in eepsite webserver?
|
| Who nominated this tool? | Us
|
| Does your community have the resources to audit this tool independently? | No
|
| Does your project have funding for the audit? | No - most of our funds are in volatile BTC and at this time are insufficient to fund an extensive audit.
|
| Has this project been audited in the past, either by us or anyone else? | No
|
Project Selection
Threat Model
| Criterion | Our response | Do we fulfil this?
|
| Does the tool have a threat model? | Yes (Needs reviewing) - http://www.i2p2.i2p/how_threatmodel | Yes
|
| Does the tool have one or more clearly defined use contexts? | Maybe? I2P is an anonymous overlay network, and has many potential use contexts. | ?
|
| Does the threat model follow a clearly established methodology? | Needs research |
|
| Is the threat model formally specified? | Define "formal" | Probably not formal enough for them
|
User Experience
| Criterion | Our response | Do we fulfil this?
|
| Does the user experience compromise the secure use of the tool? | No? |
|
| Has there been a professional designer involved in the tool development process? | Anonymous designers have donated their time and effort. No money has been paid towards improving the design of the I2P software UI. |
|
| Has there been user experience testing involved in the design process and if so, what? | Sampling of the opinions of users on IRC (a very small percentage of the estimated userbase). | No
|
| Is there a style guide or set of design guidelines for the tool? | | No
|
Documentation
| Criterion | Our response | Do we fulfil this?
|
| Is the tool documentation sufficient to guide the intended audience through using the tool properly? | |
|
| Is the documentation translated into the same set of languages as the tool? | A small subset of languages, and not completely. | No
|
| Is the documentation up to date, regularly maintained, and accurate? | Not entirely; not as often as it should be; reasonably. | No
|
| Does the documentation correctly describe the security caveats and use cases of the tool? | Yes |
|
| Does the documentation make clear security claims, and are those claims supported by the tool? | Yes? - We need a review to find out  |
|
| For tools intended for end-users, is there a set of introductory documentation for inexperienced users? | No |
|
Audience and Adversary Definition
| Criterion | Our response | Do we fulfil this?
|
| Is the tool actively designed with the needs of at-risk users in mind? | Yes | Yes
|
| Does the project understand who their users are? | Yes, in general, however specifics are difficult to gather due to the nature of anonymity tools. | Yes
|
| Does the project understand who their user's adversaries are? | Yes, in general, however specifics are difficult to gather. | Yes
|
| Is the tool actively designed with their user's adversary's capabilities in mind? | Yes, in general, however we could always do better. | Yes
|
| Is the tool being used for contexts outside of those that it was designed for? | Probably. At least one user uses I2P as a dynamic DNS with NAT traversal capability. I2P is an open source tool that is modified and packaged by several third parties. |
|
| Was the tool designed with a realistic awareness of the needs of its intended user community? | Yes, in general. See e.g. http://www.i2p2.de/_static/pdf/i2p_philosophy.pdf and old meeting logs. | Yes
|
Development Process Transparency
| Criterion | Our response | Do we fulfil this?
|
| Does the project plan development in public fora (such as a mailing list or IRC channel)? | Yes: IRC2P and freenode/#i2p-dev, http://zzz.i2p, http://lists.i2p2.i2p (primarily IRC and zzz.i2p) | Yes
|
| Does the project have an accurate roadmap that is up to date and has a history of use? | http://trac.i2p2.i2p/wiki/Roadmaps/1.0 would be the most up-to-date; http://www.i2p2.i2p/roadmap and http://www.i2p2.i2p/todo also exist. None of these have a recent history of use. | No
|
| Does the project have a public bug tracker? Has the project used their bug tracker over time and kept it accurate? | Yes to both. While we can't claim we have always kept it perfectly accurate, we have kept it current and accurate in recent years and plan to continue doing so. | Yes
|
| Does the project have a public source repository that is in use for mainline development? | Monotone (see e.g. http://viewmtn.meeh.i2p/ and public repos mtn://mtn.i2p2.de and mtn://mtn.i2p-projekt.i2p); See also git mirrors at https://github.com/i2p/i2p.i2p and http://git.repo.i2p/w/i2p.i2p.git | Yes
|
| Has the project added new developers or other volunteers over the life of the project? | Yes | Yes
|
Vulnerability Response Process Maturity and Transparency
| Criterion | Our response | Do we fulfil this?
|
| Does the project have documented criteria for determining what is a security issue? | No |
|
| Does the project have a documented process for classifying the impact of security vulnerability reports? | No Define or set up | Yes
|
| Does the project have a documented response process for security vulnerability reports? | Partial. Documented at http://zzz.i2p/topics/780 | No
|
| What is the project history of responding to security incidents and is it documented? | Generally fixed in the next release. Release schedule is accelerated if necessary. Our typical release cycle is 6-10 weeks, or about 7 releases per year. History is documented at http://zzz.i2p/forums/13 |
|
| Does the project have an internal responsible disclosure policy and is it used? | Yes to both. Documented at http://zzz.i2p/topics/780 and we generally disclose issues in release notes, see http://www.i2p2.de/announcements - We also use the project news feed which is delivered to users in the tool, as well as other channels such as Twitter and http://forum.i2p/ | Yes
|
| What timeline does the project have around responding to vulnerabilities? | Next release. Release schedule is accelerated if necessary. Our typical release cycle is 6-10 weeks, or about 7 releases per year. |
|
Project License and IP Disposition
| Criterion | Our response | Do we fulfil this?
|
| Does the project have a license on the Open Source Initiative's list of free software licenses? | Several - http://www.i2p2.i2p/licenses | Yes
|
| Is the project functionally open source such that the PRB could independently fix vulnerabilities without project team cooperation if it became necessary? This includes situations where, for example, a free software project depends on a proprietary library. | | Yes?
|
| Is the project aware of any potential patent or copyright issues with the project that would limit distribution of any part of their project or interfere with the audit process? | No |
|
Privacy and Terms of Service Disposition
| Criterion | Our response | Do we fulfil this?
|
| To what degree does the project (as opposed to tool) come into contact with confidential information? | The I2P network operates on is that there are no "trusted" routers/servers, so the project has no direct contact with confidential information. Some router statistics are publicly published to the netDB for diagnostic purposes, and users sometimes post potentially-deanonymizing information as part of bug reports. What about IPs, router IDs etc. in website logs from updates? - In-network update, also anyone can set up an update server | Yes?
|
| Does the project understand what data they gather about their users and what its privacy and security impacts are? | Check this |
|
| What do project policies permit the project to do with the data they gather? | Check this |
|
| What attitude toward users do any privacy or terms of service statements present? | Not applicable? |
|
Project Continuity
| Criterion | Our response | Do we fulfil this?
|
| Does the project have an active developer base? | Yes | Yes
|
| Does the project have a meaningful revenue or funding model sufficient to cover its costs in the long term? | Donations cover server costs and provide for bounties; many services are run by volunteers. See http://www.i2p2.i2p/halloffame for revenue details. | Yes?
|
| Does the project have an accurate roadmap that is up to date and has a history of use? | See Development Process Transparency above.
|
| Does the project have a public bug tracker? Has the project used their bug tracker over time and kept it accurate? | See Development Process Transparency above.
|
| Does the project have a public source repository that is in use for mainline development? | See Development Process Transparency above.
|
| Is the project documentation up to date in all the languages the project supports? | No, but the translation tagging of the website revamp will enable more accurate coverage. | No
|
| Does the project have a test framework? | Both JUnit and ScalaTest | Yes
|
| Does the project have significant regression testing coverage? | About 30% - see http://jenkins.killyourtv.i2p/job/cobertura/ | No
|
Project Impact
| Criterion | Our response | Do we fulfil this?
|
| How large is the project's user base? | Impossible to know for sure. Our best estimate is 20-30K daily unique users and about 40K monthly uniques. | Yes
|
| Does this project benefit an at-risk population directly or indirectly? | Directly | Yes
|
| Are there any alternatives for this functionality on the platforms it serves? | Tor provides hidden services, but unmaintained and tangential to Tor's target functionality. |
|
| Is this tool recommended by trainers or others in the field? | Unknown |
|
| How security-critical is the tool's functionality? | Being an anonymous network, other tools are dependent on its security. |
|
| Is this project infrastructure that other tools depend on? | Yes, e.g. eepsites, torrent software, http://nightweb.net | Yes
|
| What does the project's growth curve look like? | Exponential - see http://stats.i2p/cgi-bin/total_routers_year.cgi | ?
|
| Is this tool localized for significant at-risk populations? | We have translations for Arabic and Chinese (among others). | Yes?
|
| Is localization applied consistently? | Localization of the routerconsole is mostly done via gettext. Inconsistencies do occur in the separate-page translations. | Maybe?
|
Project Auditing Need
| Criterion | Our response | Do we fulfil this?
|
| Has the project been audited before, and if so how code base changed since the previous audit? | No | Yes?
|
| Are their significant known security concerns or has the project had public exploit(s)? | Future cryptographic weakness (see #856). No known public exploits of I2P itself ( Check this). |
|
| Is this project implicated in the harm of an at-risk population? | No | Yes
|
| Is the project written in a high-risk language like C? | No (Java) | Yes
|
| Is the project's development team relatively inexperienced, especially with security? | The team has a wide range of technical training and experience, ranging from college students to those with 30+ years of engineering / programming background. No active developers have indicated having any formal security background or education. | Yes?
|
