diff --git a/docs/html/index.md b/docs/html/index.md index ab0b40dc180..dad5d94f2bf 100644 --- a/docs/html/index.md +++ b/docs/html/index.md @@ -23,7 +23,7 @@ cli/index :hidden: development/index -ux_research_design +ux-research-design/index news Code of Conduct GitHub diff --git a/docs/html/ux-research-design/contribute.md b/docs/html/ux-research-design/contribute.md new file mode 100644 index 00000000000..60b982c856e --- /dev/null +++ b/docs/html/ux-research-design/contribute.md @@ -0,0 +1,24 @@ +# How to Contribute + +## Participate in UX Research + +It is important that we hear from pip users so that we can: + +- Understand how pip is currently used by the Python community +- Understand how pip users _need_ pip to behave +- Understand how pip users _would like_ pip to behave +- Understand pip’s strengths and shortcomings +- Make useful design recommendations for improving pip + +If you are interested in participating in pip user research, please [join pip’s user panel](https://mail.python.org/mailman3/lists/pip-ux-studies.python.org/). + +## Test New Features + +You can help the team by testing new features as they are released to the community. + +## Report and Work on UX Issues + +If you believe that you have found a user experience bug in pip, or you have ideas for how pip could be made better for all users, please file an issue on the [pip issue tracker](https://github.com/pypa/pip/issues/new). + +You can also help improve pip’s user experience by [working on UX issues](https://github.com/pypa/pip/issues?q=is%3Aissue+is%3Aopen+label%3A%22K%3A+UX%22). Issues that are ideal for new contributors are marked with “[good first issue](https://github.com/pypa/pip/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)”. Explore the +[UX Guidance](guidance) if you have questions. diff --git a/docs/html/ux-research-design/guidance.md b/docs/html/ux-research-design/guidance.md new file mode 100644 index 00000000000..188930a9e62 --- /dev/null +++ b/docs/html/ux-research-design/guidance.md @@ -0,0 +1,412 @@ +# UX Guidance + +This section of the documentation is intended for contributors who wish to work on improving pip's user experience, including pip's documentation. + +## What is User Centered Design? + +User-centered design (UCD) or human-centered design (HCD) is an iterative process in which design decisions are informed by an understanding of users and their needs. There are many terms used to describe this type of work; in this document we will use "user experience (UX) research and design". + +For the pip project, UX research and design can be used to: + +- Develop a deeper understanding of pip's users, the context in which they use pip and the challenges that they face +- Inform the design of new or existing pip features, so that pip us more usable and accessible. This may include improving pip's output (including error messages), controls (e.g. commands and flags) and documentation +- Help pip's development team prioritize feature requests based on user needs + +At a high level, the UX research and design process is comprised of: + +1. **[Research](#conducting-research-for-pip)**, where a variety of techniques are used (e.g.[surveys](#surveys) and [interviews](#interviews)) to learn about users and what they want from the tools they use +2. **[Design](#user-interface-design)**, where solutions are proposed to response to the research conducted. UX research and design is conducted iteratively, with design proposals or prototypes tested with users to validate that they are effective in meeting users' needs. Often, it is necessary to complete several cycles of research, design and validation to find a solution that works: + +![Graphic showing an iterative process of Research, Make (Design), Validate, around user goals and needs.](https://user-images.githubusercontent.com/3323703/124515613-c5bae880-ddd7-11eb-99d6-35c0a7522c7a.png) + +For more information on how this process has been applied to the pip project, see [research results](research-results/index). + +See also: + +- [Introduction to user centered design from the interaction design foundation](https://www.interaction-design.org/literature/topics/user-centered-design) +- [User-Centered Design Basics from usability.gov](https://www.usability.gov/what-and-why/user-centered-design.html) +- [User-centered design articles and videos from Nielson Norman Group](https://www.nngroup.com/topic/user-centered-design/) + +## Conducting Research for pip + +User research can be used to answer a few different types of questions: + +- _Understanding the context generally_ — e.g. how is pip used by people? What different environments and contexts is pip used in? +- _Understanding the users more broadly_ — e.g. who uses pip? How much experience do they have typically? How do they learn how to use pip? Are there any common characteristics between pip users? How diverse are the needs of pip's users? +- _Evaluating a specific need or challenge_ — e.g. how are pip users encountering a given issue? When does it come up? Do pip users regularly encounter this issue? How would a new feature address this issue? + +During the research process, it is important to engage users for input, and incorporate their feedback into decision making. + +Input and feedback from users is as valuable to an open source project as code contributions; end users may not be ready yet to submit a pull request or make fixes into the code directly, but their feedback can help to shape pip's priorities and direction. + +There are many ways to engage users in open source projects, and sometimes input from community members can feel overwhelming! Providing a structure, such as surveys and interviews, can make it easier to collect and understand feedback. Some examples of how to engage users are: + +- _Surveys_ — good for targeted feedback about specific issues and broad community context and understanding +- _Interviews_ — good for in-depth conversations to understand or explore a topic +- _Testing_ — good to evaluate an issue or validate a design idea +- _Open issue queues_ (e.g. GitHub issues) & support ticket systems — great data source to understand common challenges +- _Forums or discussion tools_ — great data source to understand common challenges or engage broader community in open discussion +- _Conferences and events_ — great opportunity to go lightweight interviews or testing of specific features + +When running [UX research on pip in 2020](research-results/index), we found that surveys and interviews were particularly useful tools to engage with pip's users. Some general guidelines, as well as pip-specific recommendations are below. + +### Surveys + +Surveys are great for collecting broad, large scale input, e.g. learning more about pip's user community as a whole, or for getting targeted feedback about a specific issue. + +Surveys can also be leveraged to get in-situ feedback with early releases of new tools, e.g. prompting users on the command line if they are using a beta version of a feature or asking people for feedback on a documentation page. + +As an example, in 2020, the pip UX team published several surveys to learn about pip and pip's users. This included: + +- Understanding 'who uses pip' +- Collecting feedback about pip's documentation +- Collecting feedback about pip's beta release of the 2020 dependency resolver +- Asking users how specific parts of pip's 2020 dependency resolver should behave + +A full list of the surveys published in 2020 and their results [can be found here](research-results/index). + +#### Designing Surveys + +When designing surveys, it is important to first establish what you want to learn. It can be useful to write this down as research-results/index questions. Example pip research-results/index questions [can be found here](https://github.com/pypa/pip/issues/8518). + +If you find that your topic is large, or you have many research-results/index questions, consider publishing several separate surveys, as long surveys risk a low response / high dropoff rate. + +Below is a brief guide to building a survey for pip: + +
    +
  1. + Introduce your survey
    + Explain the motivation for the survey, or (for surveys about pip's behaviour) set the scene with a scenario. +
    2. +
  2. + Design your questions
    +
      +
    • + Limit the number of questions you ask to avoid a low response rate. A good rule of thumb is: 3-4 questions about the specific topic, 2-3 questions about users level of experience / what they use Python or pip for.
      + When asking about years of experience use the following groupings as options: +
        +
      • < 1 Year
        • +
      • 1-3 Years
        • +
      • 4-6 Years
        • +
      • 7-10 Years
        • +
      • 11-15 Years
        • +
      • 16+ Years
        • +
      +
      • +
    • + Use closed questions with a fixed number of possible responses (e.g. yes/no, multiple choice, checkboxes, or likert scale) for measuring behaviour, opinion or preferences +
      • +
    • + Use open questions to learn about reasoning. If you are using a lot of closed questions in your survey, it is useful to include some open questions to "fish" for less expected answers - e.g. asking a user "why?" they chose a particular option +
      • +
    +
    3. +
  3. + Pilot your survey and modify it based on feedback
    + This could be as simple as sharing it with 1-2 people to see if it makes sense. +
    4. +
  4. + Determine where to do outreach
    + Establish who you want to hear from and where you should post the survey. Are there community members or groups that can help you reach more people?
    +
      +
    • Does the survey need to be translated into other languages to reach a broader set of the community?
      • +
    • Are you able to compensate people for their time?
      • +
    • Do participants want to be acknowledged as contributors?
      • +
    +
    5. +
  5. + Launch and promote your survey
    + See survey and interview outreach for recommendations on how to do outreach for pip based on the UX research-results/index conducted in 2020. +
    6. +
+ +#### Survey Case Study + +The process described above was followed in 2020, when we wanted to establish whether pip [should install packages with conflicting dependencies](https://github.com/pypa/pip/issues/8452). + +First, we introduced the purpose of the survey, with a scenario: + +![survey introduction with scenario with packages that conflict](https://user-images.githubusercontent.com/3323703/124516502-b046be00-ddd9-11eb-830c-62b8a6fb6182.png) + +Next, we asked a closed question to establish what the user prefers: + +![survey question asking whether pip should allow users to install packages when there are conflicting dependencies](https://user-images.githubusercontent.com/3323703/124516576-e5eba700-ddd9-11eb-8baf-e07773e75742.png) + +Following this, we qualified the response with an open question: + +![survey question asking respondents why pip should allow users to install packages with conflicting dependencies](https://user-images.githubusercontent.com/3323703/124516646-129fbe80-ddda-11eb-9c8a-da127f19fccd.png) + +This was followed by further questions about workarounds, syntax and behaviour preferences. + +Finally, we asked survey participants about themselves, including how much Python experience they have, and what they use Python for. This was to find out if different types of Python users answered the questions differently. + +This survey was shared with the pip team and improved several times, before it was published and promoted using a variety of [outreach channels](#survey-and-interview-outreach). + +In total, we received 415 responses, with [clear results](research-results/override-conflicting-dependencies) that helped us to make strong recommendations on how to move forward with this feature. + +#### Analysing Survey Results + +Surveys are particularly useful for being able to quickly understand trends from a larger population of responses. If your questions are designed well, then you should be able to easily aggregate the data and make statements such as: `X% of respondents said that Option B was the best option.` + +#### Contextualizing the Responses + +It's important to remember that the responses to your survey will be biased by the way that you did outreach for your survey, so unless you can be sure that the people who responded to your survey are representative of all of your users, then you need to be sure to contextualize the results to the participants. Within your survey responses it can be helpful to see if there is variation in the responses by different aspects of your users or your user community, e.g. + +- By experience level — Are responses consistent across experience level or do they vary? E.g. Do newer or more junior experience users have different responses, needs or challenges? +- By background/context — Are responses consistent across background or context? E.g. Do users in a corporate context have similar responses to hobbyist/independent users? Do data analysts have similar responses to software engineers? + +#### How many responses is enough? + +It depends! This is a hard question to answer in research like this — Traditional statistics would suggest that "enough" depends on the total population you need the survey to represent. In UX research, the answer tends to be more around when you see variation in responses level out, and so it's more about signals and trends in the data. + +If you are finding that there aren't patterns in the data, it might mean that your questions weren't clear or provided too many options, or it might mean that you need to reach out to more people. + +See also: + +- [28 Tips for Creating Great Qualitative Surveys from Nielson Norman Group](https://www.nngroup.com/articles/qualitative-surveys/) +- [Open vs. Closed Questions in User Research from Nielsen Norman Group](https://www.nngroup.com/videos/open-vs-closed-questions/) +- [Survey questions 101: over 70 survey question examples + types of surveys and FAQs - from HotJar](https://www.hotjar.com/blog/survey-questions/) + +### Interviews + +Interviews are a great way to have more in-depth conversations with users to better understand or explore a topic. Unlike surveys, they are not a great way to understand overall patterns, as it is hard to engage with a large number of people due to the time involved. It can be particularly useful to plan around conferences and events as a way to connect with many users in a more informal setting. + +#### Designing Interviews + +As with surveys, it's important to establish what you want to learn before you begin. + +Often, interviews are conducted with a script; this helps the interview run smoothly by providing some structure. However, it is also ok to go "off script" if the conversation is moving in an interesting or insightful direction. + +Below is a brief guide to running an interview for pip: + +
    +
  1. + Write your script
    + This should include an introduction that sets the scene for the participant, explaining what the interview is about, how you (or observers) will take notes, how long it will take, how their feedback will be used (and shared) and any other pointers you want to share.
    + Next, design your questions. Limit the number of questions, so that you have enough time to cover key points and the interview does not run for too long. Like in surveys, a good rule of thumb is 2-3 questions about users' level of experience, and what they use Python/pip for, plus 3-4 questions about the specific topic.
    + There are four different types of interview questions: +
      +
    1. + Descriptive — This type of question gives you concrete, specific stories and details. It also helps your interviewee "arrive" at the interview, resurfacing their relevant experiences and memories. E.g.
      +
        +
      • Tell me about a time…
        • +
      • Tell me about the first time…
        • +
      • Tell me about the last time…
        • +
      • Tell me about the worst/best time…
        • +
      • Walk me through how you…
        • +
      +
      2. +
    2. + Reflective — These questions allow the interviewee to revisit and think more deeply about their experiences. Helping the interviewee reflect is at the heart of your interview. Don't rush – give them lots of space to put their thoughts together.
      +
        +
      • What do you think about…
        • +
      • How do you feel about…
        • +
      • Why do you do…
        • +
      • Why do you think…
        • +
      • What effects did it have when…
        • +
      • How has ... changed over time?
        • +
      +
      3. +
    3. + Clarifying — This type of question gives interviewees the opportunity to expand on key points. Skillful clarifying questions also let you subtly direct the interviewee's storytelling towards the areas you find most intriguing and relevant.
      +
        +
      • What do you mean when you say…
        • +
      • So, in other words…
        • +
      • It sounds like you're saying [...]. Is that right?
        • +
      • Can you tell me more about that?
        • +
      +
      4. +
    4. + Exploratory — These questions are an invitation to the interview-ee to think creatively about their situation, and are best left for the end of the interview. Careful, though – suggestions from a single person are rarely the answer to your design problem, and you need to be clear to them that you're just collecting ideas at this point.
      +
        +
      • How would you change…
        • +
      • What would happen if…
        • +
      • If you had a magic wand...
        • +
      +
      5. +
    +
    2. +
  2. + Pilot interview with 1-2 people & modify based on their feedback +
    3. +
  3. + Determine how to do outreach for interviews
    +
      +
    • Who do you want to be sure to hear from? Where do you need to post to contact people for interviews? Are there community members or groups that can help you reach specific people?
      • +
    • Do the interviews need to be translated into other languages to reach a broader set of the community or a specific community?
      • +
    • How will people sign up for your interview?
      • +
    • Are you able to compensate people for their time?
      • +
    • Do participants want to be acknowledged as contributors?
      • +
    +
    4. +
  4. + Start outreach!
    + See survey and interview outreach for recommendations on how to do outreach for pip based on the UX research conducted in 2020. +
    5. +
+ +Here is an example user interview script used for speaking to users about pip's documentation: + +> **Introduction** +> +> - Firstly thank you for giving me your time and for your continued involvement. +> - The purpose of this interview is to better understand how pip's documentation is perceived and used by Python community members +> - The interview will take approximately 30 minutes. If you don't understand any of the questions please ask me to repeat or rephrase. If you don't have a good answer, feel free to tell me to skip. +> - I will be taking notes. These will be shared on GitHub or the pip docs, but we will remove any identifying data to > protect your anonymity +> - Please be honest - your feedback can help us make pip better. I won't be offended by anything you have to say :) +> - (optional) Do you mind if I record this session? +> +> **Opening questions** +> +> - Can you tell me a bit about how you use Python? +> - How long have you been using pip? +> +> **Solving problems** +> +> - Can you tell me about a time you had a problem when using pip? +> - What happened? +> - What did you do? +> - Where did you go? +> - How did you resolve your problem? +> - Please go to[ https://pip.pypa.io/en/stable/](https://pip.pypa.io/en/stable/) +> - Have you ever used this documentation? +> - On a scale of 1-10 how useful was it? +> - Why? +> - Are there any projects that you use that you'd like us to look at when thinking about improving pip's docs? +> - What makes that documentation good/useful? +> +> **Conclusion** +> +> - What one thing could the pip team do to help users troubleshoot pip problems? +> - Do you have any questions? + +#### How many interviews is enough? + +This depends on the complexity of the issue you are discussing, and whether or not you feel that you have gained enough insight from the interviews you have conducted. It also depends on whether you feel you have heard from a wide enough range of people. For example, you may wish to stop interviewing only after you have heard from both expert _and_ novice pip users. + +Often, conducting just a few interviews will uncover so many problems that there is enough material to make recommendations to the team. + +#### Analyzing Interview Data + +Formal interview analysis typically uses a process called "coding" where multiple researchers review interview transcripts and label different statements or comments based on a code system or typology that has been developed to align with the research. This is a great practice and a great way to make sure that the researchers' bias is addressed as part of the process, but most teams do not have the staffing or resources to do this practice. + +Instead many smaller teams use lightweight processes of capturing interview statements into **themes**, e.g. specific topics or issue areas around needs or challenges. Interviews are also a great source for **quotes**, which can be helpful for providing an example of why something is important or when/how something comes up for users. + +Interview analysis is frequently done using sticky notes, where you can write a quote, issue or finding on a sticky note and then move the sticky notes around into clusters that can be labeled or categorized into the themes. Remotely this can be facilitated by any number of tools, e.g. digital sticky board tools like [Miro](https://miro.com/) or [Mural](https://www.mural.co/), or even kanban board tools like [Trello](https://trello.com/), [Wekan](https://wekan.github.io/) or [Cryptpad](https://cryptpad.fr/), or this can be done just with text documents or spreadsheets, using lists and categories. It can be helpful to use a [worksheet for debriefing](https://simplysecure.org/resources/interview_synthesis.pdf) at the end of each interview to capture insights and themes quickly before you forget topics from the specific interview. + +See also: + +- [User Interviews: How, When, and Why to Conduct Them from Nielson Norman Group](https://www.nngroup.com/articles/user-interviews/) +- [Interviewing Users from Nielson Norman Group](https://www.nngroup.com/articles/interviewing-users/) + +### Survey and Interview Outreach + +The following is a list of outreach platforms that the pip team used when conducting research in 2020. Some were more successful than others: + +#### Recommended: UX Research Panel + +As part of the [2020 UX Work](research-results/index), we published a form that asked people to join a research panel and be regularly contacted about surveys and interview opportunities. This is now a [mailing list that users can sign up for](https://mail.python.org/mailman3/lists/pip-ux-studies.python.org/), and will be used in an ongoing way in addition to broad public outreach. + +#### Recommended: Twitter + +We found Twitter to be a very effective platform for engaging with the Python community and drive participation in UX research. We recommend: + +1. Asking [ThePSF](https://twitter.com/ThePSF), [PyPA](https://twitter.com/ThePyPA) and [PyPI](https://twitter.com/pypi) to retweet calls for survey and interview participation +2. Asking specific individuals (who have reach within specific communities, or general followings within the Python community) to retweet. +3. Explicitly asking for retweets within tweets +4. Responding to users within Twitter + +#### Recommended: Specific Interest Groups + +We engaged with the [PyLadies](https://pyladies.com/) community via their [Slack channel](https://slackin.pyladies.com/) to drive more participation from women using pip, as we found this demographic more difficult to reach via other channels + +#### Recommended: Conference Communities + +Due to the 2020 Global Pandemic we were unable to engage with users via PyCon (or other regional conferences) as we would have liked. However, we would still recommend this channel as a fast and insightful way to engage with large groups of interested people. + +#### Worth Exploring: Adding a prompt/path into pip's 'help' command + +We didn't have a chance to explore this opportunity, but the idea came up during workshops in December 2020 with Pypa Maintainers, and could be a great way to engage users and help point them towards opportunities to contribute. + +#### Not recommended: Forums (Discourse, etc) + +We used [discuss.python.org](https://discuss.python.org/) several times, posting to the [packaging forum](https://discuss.python.org/c/packaging/14) to ask packaging maintainers about their views on pip's functionality. Unfortunately, this was not as fruitful as we hoped, with very few responses. We found that engaging with packaging maintainers via Twitter was more effective. + +Posting surveys on Reddit was also not as useful as we had expected. If the user posting the survey or call for research participation does not have significant credit on Reddit, then the posting process itself can be challenging. Overall we did not see as much engagement in surveys or interviews come from Reddit relative to other outreach means. + +## User Interface Design + +Many people associate the term "user interface" with websites or applications, however it is important to remember that a CLI is a user interface too, and deserves the same design consideration as graphical user interfaces. + +Designing for pip includes: + +- Designing pip's _input_ - establishing the best way to group functionality under commands, and how to name those commands so that they make sense to the user +- Writing pip's _output_ - establishing how pip responds to commands and what information it provides the user. This includes writing success and error messages. +- Providing supplemental materials - e.g. documentation that helps users understand pip's operation + +### Design Principles / Usability Heuristics + +There are many interaction design principles that help designers design great experiences. Nielsen Norman's [10 Usability Heuristics for User Interface Design](https://www.nngroup.com/articles/ten-usability-heuristics) is a great place to start. Here are some of the ways these principles apply to pip: + +- Visibility of system status: ensure all commands result in clear feedback that is relevant to the user - but do not overload the user with too much information (see "Aesthetic and minimalist design") +- Consistency and standards: when writing interfaces, strive for consistency with the rest of the Python packaging ecosystem, and (where possible) adopt familiar patterns from other CLI tools +- Aesthetic and minimalist design: remove noise from CLI output to ensure the user can find the most important information +- Help users recognize, diagnose, and recover from errors: clearly label and explain errors: what happened, why, and what the user can do to try and fix the error. Link to documentation where you need to provide a detailed explanation. +- Help and documentation: provide help in context and ensure that documentation is task-focussed + +#### Additional Resources + +- [Command Line Interface Guidelines](https://clig.dev) +- [10 design principles for delightful CLIs](https://blog.developer.atlassian.com/10-design-principles-for-delightful-clis/) + +### Design Tools + +Tools that are frequently used in the design process are personas and guidelines, but also wireframing, prototyping, and testing, as well as creating flow diagrams or models. + +#### Personas + +_For a more in-depth overview of personas and using them in open source projects, this [resource from Simply Secure](https://simplysecure.org/blog/personas) may be helpful._ + +Personas are abstractions or archetypes of people who might use your tool. It often takes the form of a quick portrait including things like — name, age range, job title, enough to give you a sense of who this person is. You can capture this information into a [persona template](https://simplysecure.org/resources/persona-template-tech.pdf) and share them with your open source community as a resource see [examples from the Gitlab UX Team](https://about.gitlab.com/handbook/marketing/strategic-marketing/roles-personas/). + +Personas are particularly useful to help ground a feature design in priorities for specific needs of specific users. This helps provide useful constraints into the design process, so that you can focus your work, and not try to make every feature a swiss army knife of solutions for every user. + +In 2020, the pip UX team developed the following personas for the pip project: + +- Python Software User +- Python Software Maker +- Python Package Maintainer + +An in-depth write up on how the pip personas were created, and how they can be applied to future pip UX work can be [found here](research-results/personas). + +#### Prototyping + +In any UX project, it is important to prototype and test interfaces with real users. This provides the team with a feedback loop, and ensures that the solution shipped to the end user meets their needs. + +Prototyping CLIs can be a challenge. See [Creating rapid CLI prototypes with cli-output](https://www.ei8fdb.org/thoughts/2020/10/prototyping-command-line-interfaces-with-cli-output/) for recommendations. + +#### Copywriting Style Guides + +Given pip's interface is text, it is particularly important that clear and consistent language is used. + +The following copywriting Style Guides may be useful to the pip team: + +- [Warehouse (PyPI) copywriting styleguide and glossary of terms](https://warehouse.readthedocs.io/ui-principles.html#write-clearly-with-consistent-style-and-terminology) +- Firefox: + - [Voice and Tone](https://meet.google.com/linkredirect?authuser=0&dest=https%3A%2F%2Fdesign.firefox.com%2Fphoton%2Fcopy%2Fvoice-and-tone.html) + - [Writing for users](https://meet.google.com/linkredirect?authuser=0&dest=https%3A%2F%2Fdesign.firefox.com%2Fphoton%2Fcopy%2Fwriting-for-users.html) +- [Heroku CLI](https://devcenter.heroku.com/articles/cli-style-guide) (very specific to Heroku's CLI tools) +- [Redhat Pattern Fly style guide](https://www.patternfly.org/v4/ux-writing/about) +- [Writing for UIs from Simply Secure](https://simplysecure.org/blog/writing-for-uis) + +### General Resources + +- Heroku talk on design of their CLI tools ([video](https://www.youtube.com/watch?v=PHiDG-_XoRk) transcript) +- [Simply Secure: UX Starter Pack](https://simplysecure.org/ux-starter-pack/) +- [Simply Secure: Feedback Gathering Guide](https://simplysecure.org/blog/feedback-gathering-guide) +- [Simply Secure: Getting Quick Tool Feedback](https://simplysecure.org/blog/design-spot-tool-feedback) +- [Internews: UX Feedback Collection Guidebook](https://globaltech.internews.org/our-resources/ux-feedback-collection-guidebook) +- [Simply Secure: Knowledge Base](http://simplysecure.org/knowledge-base/) +- [Open Source Design](https://opensourcedesign.net/resources/) +- [Nielsen Norman Group](https://www.nngroup.com/articles/) +- [Interaction Design Foundation](https://www.interaction-design.org/literature) diff --git a/docs/html/ux-research-design/index.md b/docs/html/ux-research-design/index.md new file mode 100644 index 00000000000..0d9efd07ac4 --- /dev/null +++ b/docs/html/ux-research-design/index.md @@ -0,0 +1,15 @@ +# UX Research & Design + +```{toctree} +:hidden: + +contribute +guidance +research-results/index +``` + +Welcome to pip’s UX research and design documentation. The purpose of this section of the documentation is to: + +- [Identify where new contributors can participate in or lead UX research and design activities](contribute) +- [Share pip UX guidelines](guidance), including an introduction to User Centered Design practices, and how they can be applied to the pip project +- Share [results of user research](research-results/index) that the pip team has already conducted diff --git a/docs/html/ux-research-design/research-results/about-our-users.md b/docs/html/ux-research-design/research-results/about-our-users.md new file mode 100644 index 00000000000..81a0878d9fe --- /dev/null +++ b/docs/html/ux-research-design/research-results/about-our-users.md @@ -0,0 +1,291 @@ +# About pip's Users + +## Problem + +We want to understand users' background, their cultural environment, and how they experience the world, so that we can find better ways to serve them. + +[Skip to recommendations](#recommendations) + +## Research + +To develop our understanding about pip's users, we published a "Who uses pip?" survey that asked users about: + +- Their location in the world +- Their spoken language +- If they identified as members of an underrepresented group in the Python community +- Their disabilities and if those disabilities affected their usage of pip + +## Results + +164 people responded to the survey, with 40% of these coming from English speaking countries. 80% of participants came from Europe or North America. + +Approx. 60% did not identify as members of an underrepresented group. The majority of participants who did identify as underrepresented did so for gender reasons. + +The majority of participants (94%) responded that they did not have a disability. Of those that did have a disability, the majority were cognitive disabilities (Attention Deficit Hyperactivity Disorder aka ADHD, Autism, Aspergers, Dyslexia) or a hearing disability. + +### Participant Demographics + +#### Location + +The majority of participants came from North America and Western Europe. Participation from pip users in Africa, Asia, and the Middle-East was low. + +![Map of world showing distribution of participants as per table below](https://i.imgur.com/U2MYiK7.png) + +Fig. X: Global distribution of pip research participants. + +| Country Name | Number of participants | +| ------------------------ | ---------------------- | +| United States of America | 42 | +| United Kingdom | 17 | +| France | 12 | +| Germany | 11 | +| Canada | 10 | +| Netherlands | 8 | +| Spain | 6 | +| Switzerland | 5 | +| Nigeria | 4 | +| India | 4 | +| Czech Republic | 4 | +| Argentina | 4 | +| Sweden | 3 | +| Australia | 3 | +| Ukraine | 2 | +| Taiwan | 2 | +| Russia | 2 | +| Greece | 2 | +| Colombia | 2 | +| Chile | 2 | +| Brazil | 2 | +| Belgium | 2 | +| Uganda | 1 | +| Turkey | 1 | +| Singapore | 1 | +| Serbia | 1 | +| Norway | 1 | +| Luxembourg | 1 | +| Japan | 1 | +| Italy | 1 | +| Israel | 1 | +| Ireland | 1 | +| Hungary | 1 | +| Ghana | 1 | +| Finland | 1 | +| Bulgaria | 1 | +| Austria | 1 | +| Total | 164 | + +#### Participant's First Language + +Even though the research was carried out mainly in English, 51% of participants spoke languages other than English. + +| What spoken language do you feel is your first? | Number of participants | +| ----------------------------------------------- | ---------------------- | +| English | 79 | +| French | 18 | +| Spanish | 12 | +| German | 11 | +| Russian | 5 | +| Czech | 4 | +| Italian | 4 | +| Portuguese | 3 | +| Dutch | 3 | +| Ukrainian | 2 | +| Swedish | 2 | +| Greek | 2 | +| Catalan | 2 | +| Mandarin | 2 | +| Hungarian | 2 | +| Bengali | 1 | +| Luxembourgish | 1 | +| Bulgarian | 1 | +| Romanian | 1 | +| Chinese | 1 | +| Norwegian | 1 | +| Serbian | 1 | +| Polish | 1 | +| Hebrew | 1 | +| Indonesian | 1 | +| Malayalam | 1 | + +NB: English includes British English and American English. Some participants gave more than one answer, their first answer is included here. + +### Participants who identified as underrepresented in the Python community + +We asked research participants if they identified as members of an underrepresented group within the Python community. + +The wording of this question was deliberately broad to discover participants' understanding of the term "underrepresented" - we listed gender, age, educational background, spoken language, and what they use Python for as a non-exhaustive list of examples. + +![Pie chart showing 17.9% answering - I am not sure, 22.6% answering -yes, 59.4% answering - No to the question - Do you identify as an underrepresented group in the Python community](https://i.imgur.com/ghwzxg9.png) + +Of the 22.6% that responded "Yes" the answers were classified as follows : + +| Underrepresentation category | Count | +| ---------------------------- | ----- | +| Gender | 9 | +| Cultural | 3 | +| Age | 3 | +| Immigration status | 2 | +| Neurodiversity | 3 | +| Other | 6 | +| No answer | 8 | + +NB: This question was included after the survey was published. Total participants was 106, as opposed to all other questions which had 164. + +The majority of participants did not identify as part of an underrepresented group. However, due to the small sample size these results cannot be seen as representative of the whole pip user base. + +#### Participant comments about identifying (or not) as under-represented + +Here is a sample of noteworthy comments from these different groups: + +##### Related to Gender + +> "(I am) LGBTQ/IA+" **- Participant 242608909** + +> "I am a 25 year old female Colombian developer." **- Participant 242611698** + +> "Female, 39, no computer science background whatsoever, self taught." **- Participant 242614039** + +##### Related to Culture + +> "The hispanic community is quite underrepresented in the web in general" **- Participant 242599212** + +> "I am a 1st generation Dominican-American. My parents are from the Dominican Republic." **- Participant 242769361** + +##### Related to Age + +> "Older age, I am 50 now." **- Participant 242769743** + +##### Related to Neurodiversity + +> "I'm a woman. And autistic. But the latter might not be underrepresented ;)" **- Participant 243428773** + +##### Other Noteworthy Comments + +> "Veterans who entered tech post-military" **- Participant 243524784** + +> "I'm a young white cis male, so by far not a minority in those aspects. But at the same time I'm from a third world country, Argentina, and that sometimes (and I emphasize, only sometimes) makes me feel like a minority. When participating in our local communities (Python Argentina), I feel clearly not-minority, and with the responsibility of helping minorities, trying to build a more welcoming and fair environment for them. But when I participate in the broader global community, at times I feel underrepresented, seeing it mostly guided by english-s[p]eaking people from first world countries. But if I have to choose, I would say I mostly feel not-minority, because I mostly interact with people from our local communities, where I'm not part of a minority." **- Participant 242592869** + +> "As a CIS male I conform the majoritarian group in the IT world. I'm hopeful that things are changing everywhere, and will keep changing: inclusion is getting bigger and better, more and more people are starting their careers as devs or similar, disregarding ethnicity and/or sexual orientation and that's great! And we need to keep fighting for that." **- Participant 243455292** + +### Participant Disabilities + +Disabilities - physical, motor, cognitive, hearing, speech - alter how people perceive and interact with the world around them - software included. We asked participants about their disabilities and how it affected their usage of pip. + +Understanding these disabilities is important particularly when designing pip command structures, and designing pip output. + +The majority of participants (91%) responded that they did not have a disability. Of those that told us that have a disability, the majority were cognitive disabilities (Attention Deficit Hyperactivity Disorder (ADHD), Autism, Aspergers, Dyslexia, or a hearing disability. + +#### How many participants identified as having a disability? + +| Do you self-identify as someone who has a disability? | Number of responses | +| ----------------------------------------------------- | ------------------- | +| No | 150 | +| Yes | 14 | +| Grand Total | 164 | + +#### Vision + +Participants who answered yes to this question were partially sighted. Their vision disability was not corrected by glasses, but did not significantly affect their usage of pip. + +#### Hearing + +Five participants identified as having a hearing impairment, or hearing loss. While this disability made participants lives more difficult, it did not affect their usage of pip: + +> "Being hard of hearing/impaired makes my life much harder, but so far it never has impacted my usage of pip. Perhaps because I haven‘t used parts of it that would?" **- Participant 242934019** + +> "Not at all given that everything happens by text in my console." **- Participant 243455292** + +However it did affect the way they consume pip learning materials: if video is being used for learning or support, they should have captions/subtitles/transcriptions available. + +> "any videos released, it is so helpful if there is either a) transcripts, or b) captions." **- Participant 243524784** + +#### Cognitive Disabilities + +Nine participants expressed cognitive disabilities including undefined mental health conditions, Attention Deficit Hyperactivity Disorder (ADHD), Autism, Aspergers, Dyslexia. + +These participants did not explain how their cognitive disabilities affect their usage of pip, however there are guidelines and best practices for designing for people with cognitive disabilities. + +#### Physical or Mobility Disability + +One participant responded that they had a physical or mobility disability, but did not give detail about it in relation to their usage of pip. + +### Participants use of Assistive Technology + +The term "assistive technology" (AT) is used to describe products or systems that support and assist individuals with disabilities, restricted mobility or other impairments to perform functions that might otherwise be difficult or impossible. A subset of these are used to make computer technology - hardware and software - more accessible. Common examples of AT used with computer technology are: screen readers, text-to-speech outputs/inputs. + +The majority of participants (94%) said that they have never used assistive technology. + +| Do you use assistive technology (AT) when using computers? | Number of responses | +| ---------------------------------------------------------- | ------------------- | +| No, I have never used it | 128 | +| I only use it when needed | 3 | +| I use it everyday | 1 | +| I have used it in the past, but not anymore | 4 | + +Of the eight participants who have used assistive technology, one participant uses assistive technology every day with: + +- Text-to-speech output as "text to speech allow(s) me to listen and learn when my eyes get strained." +- Speech-to-text input as they like using their "tablet and makes typing easier" +- On-screen keyboards +- Input switches/touch screens + +A further seven participants use assistive technology only as needed: + +> "I use custom display filter software to do things like colorize key lines of output automatically (to draw my eye/attention), and provide digit dilimination (I.E. help me tell 1000 and 10000 apart) when using a text console application." + +> "The standard Mac user interface design contains enough assistive technology without my needing to use any features which are specifically intended solely as assistive functions." + +> "I sometimes use it to make sure that my code will work correctly with AT." + +#### Operating systems used with assistive technology + +Participants use assistive technology across the three most popular desktop operating systems - Linux (most popular), Windows (2nd most popular), and Mac. + +![Pie chart showing breakdown of most popular operating systems for pip users using assistive technology](https://i.imgur.com/CD2ev5P.png) + +#### Assistive technology when using pip + +We asked participants how well their assistive technologies worked when they use pip. All participants using assistive technology with pip said it worked well for them. + +We received some feedback about screen readers not coping well with long output, with users experiencing difficulties accessing content at the top of the current terminal window. Therefore, commands or actions (e.g. pip help, pip install, failed builds) that generate a lot of content can be a problem for screenreader users. + +## Recommendations + +### Supporting languages other than English + +As 51% of participants speak a language other than English, we recommend that the pip team add localization support to the pip documentation and reach out to the community to find pip users who might be willing to contribute translations. Translators that have [contributed translations to PyPI](https://hosted.weblate.org/projects/pypa/warehouse/) may be a good starting point. + +If this is not possible, we recommend linking to useful resources in languages other than English from the pip documentation, as we know from our other research that users use a mixture of the official documentation, search engine searches, Stack Overflow and blogs/websites to find solutions to their problems. + +### Supporting pip users with disabilities + +Pip's operation is generally very good for users with disabilities. Being a command line application there are no distracting images or ancillary content, and the user has a large amount of control on how they experience pip via customisation of interface visual preferences (to use contrasting colours, font size and type) and visual and auditory alerts. + +To better support pip's users with disabilities, the pip team should: + +- Ensure any future video or audio support materials are provided with captions +- Improve pip's output (see below) + +### Improving pip output + +Pip's output is currently too verbose, generating an unhelpful amount of output during its operation. This causes usability issues for all users - especially users with cognitive disabilities. + +Pip's output should be improved by: + +- Retaining only the information that is important to users in their current moment (e.g. at install of a package) +- Removing unimportant information from the terminal output. The information can still be logged to the log files if needed. +- Reducing the number of verbosity levels to three. Right now there are seven levels of verbosity, which is overwhelming and in no way useful. We recommend: + - Verbosity 0 - shows only what packages are to be installed, notifications identified as important about the operation, any errors and the final outcome + - Verbosity 1 - shows more detail about the packages being installed + - Verbosity 2 - shows full information which is also logged to logfiles + +## Further reading + +Designing for people with disabilities: + +- [An Introduction to inclusive design](https://www.nomensa.com/blog/2011/introduction-inclusive-design) +- [How ADHD and dyslexia teach you to do better UX design](https://themasters.io/blog/posts/how-adhd-dyslexia-teach-better-ux-design) +- [Improve User Experience by Designing with Cognitive Differences in Mind](https://noti.st/elizabethschafer/fg3BR4) +- [Designing accessible software - guidelines for different disabilities](https://ukhomeoffice.github.io/accessibility-posters/) +- [Designing for Children with ADHD: The Search for Guidelines for Non-Experts](https://uxpamagazine.org/designing_children_adhd/) (written for children however applicable generally) +- [Designing for dyslexia](https://uxplanet.org/designing-for-dyslexia-6d12e8c41cd7) diff --git a/docs/html/ux-research-design/research-results/ci-cd.md b/docs/html/ux-research-design/research-results/ci-cd.md new file mode 100644 index 00000000000..f136ddb16ab --- /dev/null +++ b/docs/html/ux-research-design/research-results/ci-cd.md @@ -0,0 +1,42 @@ +# How pip is used in interactive environments (i.e. CI, CD) + +## Problem + +We want to know about the contexts in which pip users use pip - interactively (i.e. typing pip commands at the command line terminal) and in an automated environment (i.e. as part of continuous software integration or continuous software development pipelines). + +Different contexts of use mean that users have different important and common tasks; it means when, where and how they complete these tasks are different. + +Each of these contexts bring different needs: interactive usage requires the right feedback/output at the right time, whereas an automated environment requires little or no feedback in the moment but detailed feedback after the task has finished. + +We also wanted to know what users used pip for - as part of their software development toolchain, or purely as a software installer (analogous to Ubuntu Aptitude or Mac Appstore). We also asked about their need for pip to build packages from source. + +## Research + +We created a survey and asked users to give answers to the following statements: + +- I use pip in an automated environment (e.g. CI/CD pipelines) +- I have problems with pip in CI/CD pipelines +- I use pip interactively (e.g. typing pip commands on the commandline) +- I make software and use pip as part of my software development workflow +- I use pip only to install and use Python packages +- I need pip to build software packages from source + +## Results + +Using pip interactively makes up the majority of pip usage (91%), the majority (73%) of this usage is basic usage - to only install and use Python packages. + +Half (51%) of all participants used pip in an automated environment, with only 9% having issues with pip in that automated environment. This points to a good use experience for these users. + +71% use pip as part of their software toolchain, only 29% needing pip to build from source. + +These results show that the main context of use is interactive - users either writing code, installing software at the command line and we know from other research that interactive usage has its issues e.g. pip output being too verbose. + +While it is important to provide automated environment users with a good experience, interactive mode users are being underserved. + +![Answer to question - I use pip in an automated environment](https://i.imgur.com/pLHqBpN.png) + +![Answer to question - I use pip interactively](https://i.imgur.com/8ETVMYS.png) + +91% of users said they used pip interactively. This does not preclude them from automated usage. + +![Answer to the question - What do you use Python for?](https://i.imgur.com/ySlo2Es.png) diff --git a/docs/html/ux-research-design/research-results/improving-pips-documentation.md b/docs/html/ux-research-design/research-results/improving-pips-documentation.md new file mode 100644 index 00000000000..bec2120cda6 --- /dev/null +++ b/docs/html/ux-research-design/research-results/improving-pips-documentation.md @@ -0,0 +1,531 @@ +# Improving pip's Documentation + +## Problem + +We want to establish whether or not the [official pip documentation](https://pip.pypa.io/en/stable/) helps users to solve their pip problems. We also want to identify possible improvements to the content and structure of the docs. + +[Skip to recommendations](#recommendations) + +## Research + +### Interviews + +We conducted interviews with pip users specifically discussing documentation. During these interviews we asked about: + +- Problems they had experienced while using pip, and how they solved them (with a focus on what information sources they used) +- How they rate pip's documentation, and what we could do to make the docs more useful +- What documentation (from other projects or languages) they find valuable, and why + +### Surveys + +We collected documentation feedback via two surveys: + +- In our survey that profiled pip users, we asked "What would be your ideal way of getting help with pip?" +- We also published a survey specific to pip's docs: + +![Screenshot of survey](https://i.imgur.com/dtTnTQJ.png) + +### Keyword research + +We used keyword research tools to understand what words ("keywords") people use when using search engines to troubleshoot pip problems. + +### Other research methods + +We also: + +1. Asked for volunteers to participate in a diary study, documenting their experience solving pip problems. Unfortunately this was not completed due to lack of interest from the community. +2. Asked for user feedback on the pip documentation site: + ![screenshot of user feedback mechanism on pip docs](https://i.imgur.com/WJVjl8N.png) + Unfortunatly, we did not gather any useful feedback via this effort +3. [Installed analytics on the pip docs](https://github.com/pypa/pip/pull/9146). We are waiting for this to be merged and start providing useful data. + +## Results + +In total, we: + +- Conducted 5 user interviews about pip's documentation +- Received 141 responses to the question "What would be your ideal way of getting help with pip?" +- Received 159 responses to the documentation survey + +In general, we found that pip's documentation is underutilized by the community, with many users not knowing that it exists. Instead, most users turn to common tools (Google, Stack Overflow) to solve their pip problems. + +In response to the question "When you have a problem using pip, what do you do?" (multiselect): + +- 81.9% of respondents Google it +- 56.9% of respondents search or ask on Stack Overflow +- 33.8% of respondents use pip help from the command line +- **25.6% of respondents go to the pip docs** +- 20.6% of respondents go the the Python Packaging User Guide +- 8.1% of respondents ask on a forum, community board, or chat channel + +![screenshot of survey results](https://i.imgur.com/qlt1b4n.png) + +Based on survey results, users find pip's docs: + +- Marginally more useful than not useful +- Marginally more clear than unclear +- Not opinionated enough + +Common feedback that emerged from both surveys and user interviews includes: + +- The documentation performs poorly in search engine results +- The style and layout is dated (note: this feedback was collected before the [new theme was deployed](https://github.com/pypa/pip/pull/9012)) +- There is not enough guidance/examples on how to resolve common problems, or achieve specific goals +- The documentation information architecture is difficult to navigate (the monolithic structure of the user guide is a problem) and does not prioritise the most useful content +- There should be more instructions specific to each user's different situation (e.g. what operating system they are using) +- The scope of the documentation is unclear +- The documentation should recognise that pip exists within an ecosystem of other packaging tools +- ["There should be one-- and preferably only one --obvious way to do it."](https://www.python.org/dev/peps/pep-0020/) - i.e. the documentation should provide stronger recommendations + +While some users mentioned that video would be helpful, more said that video was too long, or inappropriate for the kind of problems they experience using pip. + +Some users mentioned that in person support, forums or chat would be helpful, with many unaware of existing support / community channels. + +Several users also noted that improving pip's error messages would reduce the need for better documentation. + +From our keyword research we identified seven _query types_: "about pip", "install pip", "uninstall pip" "update pip", "using pip", "errors", and "other". + +
See keyword research results + +### About pip + +- what is pip +- what is pip in python +- what is pip python +- what does pip mean +- what does pip stand for +- what does pip stand for python +- pip meaning + +### Install pip + +- get pip +- python install pip +- install pip +- installing pip +- how to install pip python +- how to install pip +- how to download pip +- how to get pip +- how to check if pip is installed +- install pip mac +- how to install pip on mac +- install pip on mac +- install pip linux +- how to install pip linux +- how to install pip on linux +- how to install pip in ubuntu +- how to install pip ubuntu +- install pip ubuntu +- ubuntu install pip +- pip windows +- install pip windows +- pip install windows +- how to install pip windows +- how to install pip in windows +- how to install pip on windows +- how to pip install on windows +- how to install pip on windows 10 +- how to run pip on windows + +### Uninstall pip + +- how to uninstall pip +- uninstall pip +- pip uninstall + +### Update pip + +- how to update pip +- how to upgrade pip +- pip update +- pip upgrade +- upgrade pip +- how to upgrade pip on windows + +### Using pip + +- how to use pip +- how to use pip install +- how to pip install +- how to use pip python +- how to install with pip +- how to run pip +- python how to use pip +- pip install requirements.txt +- pip requirements.txt +- pip freeze +- pip update package +- pip install specific version +- pip upgrade package +- pip uninstall package + +### Errors + +- no module named pip +- pip command not found +- pip is not recognized +- 'pip' is not recognized as an internal or external command, operable program or batch file. +- -bash: pip: command not found +- pip is not recognized as an internal or external command +- pip install invalid syntax + +### Other + +- how to add pip to path +- how to check pip version +- how does pip work +- where does pip install packages +- pip vs pip3 +- where is pip installed + +
+
+ +The prevalence of "install pip" queries strongly suggests that the current installation documentation should be improved and that users are searching for solutions specific to their operating system. + +The "about pip" queries also suggest that beginners would benefit from documentation that better explains pip basics - e.g. what pip is and what it does. + +## Recommendations + +Based on our research, we recommend that the pip team: + +- Revise the structure of the documentation: + - Break monolithic pages into standalone pages on different subjects, with appropriate meta tags. This will help the docs appear higher in search results for the 81.9% of users who use Google to troubleshoot their pip problems. + - Prioritise most used features (see "[buy a feature](prioritizing-features)" results for guidance) +- Add a "troubleshooting" section to the documentation that addresses common questions, explains error messages and tells users where they can find more help +- Provide more context about pip's role in the Python packaging ecosystem by: + - Introducing packaging concepts that users need to understand in order to use pip + - Explaining pip's role/scope within the packaging ecosystem + - Comparing pip to other tools +- Develop a beginner's guide that walks new pip users through everything they need to know to use pip's most basic functionality. This should include addressing concepts outside of pip's scope (e.g. how to open and use a terminal, how to set up a virtual environment), that may block users from being successful +- For each page, (where appropriate), add sections for: + - "tips and tricks" - things to know / gotchas + - "troubleshooting" - possible error messages and recommended solutions. Where appropriate, this should link to content in the troubleshooting section. + - "see also" (links to external resources - e.g. useful stack overflow questions, blog articles, etc.) +- In general, write content that: + - Is opinionated. Prioritize solutions that will work in the majority of cases, while pointing to possible edge cases and workarounds in "tips and tricks", "troubleshooting" and "see also" content + - Uses keywords to increase search results visibility + - Provides instructions for different contexts - e.g. for users on Windows, Linux, MacOSX + - Increases interlinking with external sources, including packaging.python.org + +### Suggested site map + +Based on the above user input, we have developed a proposed [site map](https://i.imgur.com/UP5q09W.png) (link opens larger format image) to help guide the redevelopment of pip's documentation in line with the above recommendations. + +![sitemap. for details see summary below](https://i.imgur.com/UP5q09W.png) + +
See notes for this site map + +#### Node 1.0: Quick reference + +_Page purpose:_ + +- To give pip users a quick overview of how to install pip, and use pip's main functionality +- To link to other (more detailed) areas of the documentation + +_Suggested content:_ + +- Quick installation guide, including how to use a virtual environment. This is necessary for user who want to install more than one Python project on their machine. +- Common commands / tasks (based on [buy a feature](prioritizing-features) data) + +--- + +#### Node 2.0: About pip + +_Page purpose:_ + +- To introduce pip to new users + +_Suggested content:_ + +- Introduce pip as a command line program +- Explain what the command line is and how to use it in different operating systems +- Explain what pip is/does, and what it stands for +- Link to packaging concepts (node 2.1) +- Explain pip's scope (e.g. to install and uninstall packages) and link to other tools (node 2.2) + +#### Node 2.1: Packaging concepts + +_Page purpose:_ + +- To introduce packaging concepts for new pip users + +_Suggested content:_ + +- What is a package? +- What types of packages are there? e.g. file types +- What is package versioning / what are requirement specifiers? (note: talk about potential dependency conflicts here) +- Where do I get packages from? +- How should I control how packages are installed on my system (e.g. virtualenv and environment isolation) +- How can I reproduce an environment / ensure repeatability? (e.g requirements files) +- What do I need to know about security? (e.g. hash checking, PyPI name squatting) +- Link to node 2.2 ("pip vs other packaging tools") + +#### Node 2.2: pip vs other packaging tools + +_Page purpose:_ + +- To compare pip to other tools with the same scope +- To highlight that pip exists within a _packaging ecosystem_ and link to other packaging tools + +_Suggested content:_ + +- Compare pip to other installation tools - e.g. poetry, pipenv, conda. What are the features, pros and cons of each? Why do packaging users choose one over the other? +- Briefly introduce other packaging projects. Link to https://packaging.python.org/key_projects/ + +--- + +#### Node 3.0: Installing pip + +_Page purpose:_ + +- To help pip users install pip + +_Suggested content:_ + +- Refactor current page, emphasising pathways for different operating systems +- Add "tips and tricks", "troubleshooting" and "see also" (link to external resources) sections to provide additional help + +--- + +#### Node 4.0: Tutorials + +_Page purpose:_ + +- To provide a jumping off place into pip's tutorials + +_Suggested content:_ + +- Link to tutorials, including sub pages, where appropriate + +#### Node 4.1: Using pip to install your first package + +_Page purpose:_ + +- To help new pip users get started with pip + +_Suggested content:_ +Step by step tutorial (possibly broken into several pages) that covers: + +- Using the command line +- Installing pip (or checking pip is installed) +- Creating/activating a virtual env (use venv for this, but point to alternatives) +- Installing a package +- Showing where the package has been installed +- Deactivating/reactivating virtualenv +- Uninstalling a package + +#### Node 4.2: Advanced tutorial - using pip behind a proxy + +_Page purpose:_ + +- To help advanced pip users achieve specific goals + +_Suggested content:_ + +- Step by step tutorial for using pip behind a proxy + +NB: other advanced tutorials should be added as identified by the team and/or requested by the community. + +--- + +#### 5.0: Using pip + +_Page purpose:_ + +- To provide a jumping off point for the user guide and reference guide + +_Suggested content:_ + +- Link to each subject in the user guide +- Link to reference guide + +#### 5.1: User guide + +_Page purpose:_ + +- To provide users with specific detailed instructions on pip's key features + +_Suggested content:_ +Break down current user guide into separate pages, or pages linked by subject. Suggested order: + +- Running pip +- Installing Packages +- Uninstalling Packages +- Environment recreation with requirements files + - sub heading: "pinned version numbers" + - sub heading: "hash checking mode" +- Listing Packages +- Searching for Packages +- Installing from local packages +- Installing from Wheels +- Wheel bundles +- “Only if needed” Recursive Upgrade +- Configuration +- User Installs +- Command Completion +- Basic Authentication Credentials +- Using a Proxy Server (includes link to tutorial) +- Constraints Files +- Using pip from your program + +Where possible, each page should include: + +- "tips and tricks" for workarounds, common _gotchas_ and edge use cases +- "troubleshooting" information, linking to content in node 6.2 ("Troubleshooting error messages") where applicable +- "see also", linking to external resources (e.g. stack overflow questions, useful threads on message boards, blogs posts, etc. + +Note: the following content should be moved: + +- Fixing conflicting dependencies (move to node 6.2 - "Troubleshooting error messages") +- Dependency resolution backtracking (move to node 6.2 - "Troubleshooting error messages") +- Changes to the pip dependency resolver in 20.3 (move to node 7.0 - "News, changelog and roadmap") + +#### 5.2: Reference guide + +_Page purpose:_ + +- To document pip's CLI + +_Suggested content:_ + +- https://pip.pypa.io/en/stable/reference/ + +--- + +#### 6.0: Help + +_Page purpose:_ + +- To provide a jumping off place for users to find answers to their pip questions + +_Suggested content:_ + +- Links to + - 6.1 "FAQs" + - 6.2 "Troubleshooting error messages" + - 6.3 "Finding more help" + +#### 6.1: FAQs + +_Page purpose:_ + +- To answer common pip questions / search terms + +_Suggested content:_ + +- What is the difference between pip and pip3? +- Where does pip install packages? +- How can I check pip's version? +- How can I add pip to my path? +- Where is pip installed? +- What does pip stand for? + +See [popular questions on Stack Overflow](https://stackoverflow.com/search?q=pip&s=ec4ee117-277a-4c5d-a3f5-c921ca6c5da6) for more examples. + +#### 6.2: Troubleshooting error messages + +_Page purpose:_ + +- To help pip users solve their problem when they experience an error using pip + +_Suggested content:_ +For each (common) error message: + +- Explain what happened +- Explain why it happened +- Explain what the user can do to resolve the problem + +Note: the [ResolutionImpossible](https://pip.pypa.io/en/stable/user_guide/#fixing-conflicting-dependencies) and [dependency resolution backtracking](https://pip.pypa.io/en/stable/user_guide/#dependency-resolution-backtracking) +documentation should both be moved here. + +#### 6.3: Finding more help + +_Page purpose:_ + +- To point pip users to other resources if they cannot find the information they need within the pip documentation + +_Suggested content:_ + +- See [getting help](https://pip.pypa.io/en/stable/user_guide/#getting-help) + +--- + +#### 7.0: News, changelog and roadmap + +_Page purpose:_ + +- To share information about: + - Recent changes to pip + - Upcoming changes to pip + - Ideas for improving pip, specifically highlighting where funding would be useful + +_Suggested content:_ + +- [Changes to the pip dependency resolver in 20.3 (2020)](https://pip.pypa.io/en/stable/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020) +- Links to PSF blog posts about pip +- Link to [fundable packaging improvements](https://github.com/psf/fundable-packaging-improvements/blob/master/FUNDABLES.md) + +--- + +#### 8.0: Contributing + +_Page purpose:_ + +- To encourage new people to contribute to the pip project +- To demonstrate that the project values different _types_ of contributions, e.g. not just development +- To recognise past and current contributors + +_Suggested content:_ + +- Introduction to pip as an open source project +- Contributors code of conduct +- Recognition of the different types of contributions that are valued +- Credit list of contributors, including pip maintainers + +#### 8.1: Development + +_Page purpose:_ + +- To onboard people who want to contribute code to pip + +_Suggested content:_ + +- https://pip.pypa.io/en/stable/development/ + +#### 8.2: UX design + +_Page purpose:_ + +- To onboard people who want to contribute UX (research or design) to pip +- To share UX knowledge and research results with the pip team + +_Suggested content:_ + +- UX guidelines, and how they apply to the pip project +- Current UX initiatives (e.g. open surveys, interview slots, etc.) +- Previous research and results, including UX artifacts (e.g. personas) + +#### 8.3: Documentation + +_Page purpose:_ + +- To onboard people who want to contribute to pip's docs +- To share previous research and recommendataions related to pip's docs + +_Suggested content:_ + +- This guide +- Writing styleguide / glossary of terms - see the [Warehouse documentation](https://warehouse.readthedocs.io/ui-principles.html#write-clearly-with-consistent-style-and-terminology) for an example. + +
+ +### Future research suggestions + +To continue to improve pip's documentation, we suggest: + +- Conducting [card sorting](https://www.nngroup.com/articles/card-sorting-definition/) with pip users to establish the ideal order and grouping of pages +- Regularly reviewing the documentation analytics, to understand those pages which are most/least visited +- Regularly reviewing Stack Overflow to identify questions for the FAQ +- Setting up a mechanism for collecting user feedback while users are on the documentation site diff --git a/docs/html/ux-research-design/research-results/index.md b/docs/html/ux-research-design/research-results/index.md new file mode 100644 index 00000000000..588aef397a1 --- /dev/null +++ b/docs/html/ux-research-design/research-results/index.md @@ -0,0 +1,208 @@ +# UX Research Results + +Over the course of 2020, the pip team worked on improving pip's user experience, developing a better understanding of pip's UX challenges and opportunities, with a particular focus on pip's new dependency resolver. The [Simply Secure](https://simplysecure.org/) team focused on 4 key areas: + +- [Understanding who uses pip](https://github.com/pypa/pip/issues/8518) +- [Understanding how pip compares to other package managers, and supports other Python packaging tools](https://github.com/pypa/pip/issues/8515) +- [Understanding how pip's functionality is used could be improved](https://github.com/pypa/pip/issues/8516), and +- [Understanding how pip's documentation is used, and how it could be improved](https://github.com/pypa/pip/issues/8517) + +Some key outcomes from the 2020 work are: + +- This documentation & resource section! +- A pip UX research panel ([Sign up here!](https://mail.python.org/mailman3/lists/pip-ux-studies.python.org/)) +- New and expanded GitHub issues +- UX improvements in 2020 + - UX work supporting the dependency resolver + - Improved error messaging + - Supporting Documentation +- UX Training for the Pypa + pip maintainers + +This work was made possible through the [pip donor funded roadmap](https://wiki.python.org/psf/Pip2020DonorFundedRoadmap). + +## Outreach + +We [recruited participants](https://www.ei8fdb.org/thoughts/2020/03/pip-ux-study-recruitment/) for a user research panel that we could contact when we wanted to run surveys and interviews about pip. In total 472 people signed up to the panel, although some unsubscribed during the research period. + +At the end of the 2020 research, we asked users to opt-in to a [long-term panel](https://mail.python.org/mailman3/lists/pip-ux-studies.python.org/), where they can be contacted for future UX studies. Should the pip team wish to continue to build this panel, we recommend translating the sign-up form into multiple languages and better leveraging local communities and outreach groups (e.g. PyLadies) to increase the diversity of the participants. + +## User Interviews + +In total, we **interviewed 48 pip users**, recruited from the user panel, and through social media channels. + +During the interviews, we asked users about: + +- How they use Python +- How long they have been using pip +- Whether or not they use a virtual environment +- If and how they address security issues associated with pip +- Which pip commands they regularly use +- How they install packages with pip +- Their experience using pip list, pip show and pip freeze +- Their experience using pip wheel +- Whether or not they use other package managers, and how pip compares to their experience with these other tools +- What the pip team could do to improve pip +- Problems they have experienced while using pip, and how they solved these problems +- Their perception and use of the pip documentation +- What other technical documentation they value, and how the pip docs could take inspiration from these +- What other resources the pip team could provide to help pip users solve their problems + +## Surveys + +We **published 10 surveys** to gather feedback about pip's users and their preferences: + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TitlePurposeResults
+ Pip research panel survey + + Recruit pip users to participate in user research, user tests and participate in future surveys. See associated blog post for more information. + + 472 full sign-ups +
+ Feedback for testing the new pip resolver + + Understand use cases where the new resolver fails due to dependency conflicts. See associated blog post for more information. + + 459 responses via the feedback form, approx. 8 issues transferred to issue tracker +
+ How should pip handle conflicts with already installed packages when updating other packages? + + Determine if the way that pip handles package upgrades is in-line with user's expectations/needs. See related blog post and GitHub issue for more information. + + See write up, including recommendations +
+ Learning about our users + + Learn about pip's users, including: +
    +
  • their usage of Python and pip
    • +
  • why and how they started using Python
    • +
  • if they are living with any disabilities, and if so what effect (if any) this has on their usage of Python and pip
    • +
  • if they use assistive technologies when using Python and pip and how this work for them
    • +
  • where they get support when you have issues with pip
    • +
+ 		+ See write up +
+ Buy a pip feature + + Establish which features are most important to pip's users + + See write up +
+ Should pip install conflicting dependencies? + + Establish whether pip should provide an override that allows users to install packages with conflicting dependencies + + See write up +
+ How should pip force reinstall work? + + Establish whether or not pip force reinstall should continue to behave the way it currently does, if the functionality should be changed, or if the option should be removed + + See write up +
+ Feedback on pip search + + To establish whether or not to remove or redesign pip search. See this GitHub issue for more information. + + See write up +
+ Feedback on pip's docs + + To gather feedback on pip's docs, supplementing feedback gathered in user interviews + + See write up +
+
+ +## All Results + +```{toctree} +:maxdepth: 1 + +about-our-users +mental-models +users-and-security +ci-cd +personas +prioritizing-features +override-conflicting-dependencies +pip-force-reinstall +pip-search +pip-upgrade-conflict +improving-pips-documentation +``` + +## Read More + +- [Pip team midyear report (blog, July 2020)](https://pyfound.blogspot.com/2020/07/pip-team-midyear-report.html) +- [Creating rapid CLI prototypes with cli-output (blog, Oct 2020)](https://www.ei8fdb.org/thoughts/2020/10/prototyping-command-line-interfaces-with-cli-output/) +- [Changes are coming to pip (video)](https://www.youtube.com/watch?v=B4GQCBBsuNU) +- [How should pip handle dependency conflicts when updating already installed packages? (blog, July 2020)](https://www.ei8fdb.org/thoughts/2020/07/how-should-pip-handle-conflicts-when-updating-already-installed-packages/) +- [Test pip's alpha resolver and help us document dependency conflicts (blog, May 2020)](https://www.ei8fdb.org/thoughts/2020/05/test-pips-alpha-resolver-and-help-us-document-dependency-conflicts/) +- [How do you deal with conflicting dependencies caused by pip installs? (blog, April 2020)](https://www.ei8fdb.org/thoughts/2020/04/how-do-you-deal-with-conflicting-dependencies-caused-by-pip-installs/) +- [pip UX studies: response data (blog, March 2020)](https://www.ei8fdb.org/thoughts/2020/03/pip-ux-studies-response-data/) + +Other PyPA UX work: + +- [PyPI User Research (blog, July 2018)](https://whoisnicoleharris.com/2018/07/22/pypi-user-research.html) +- [Warehouse - The Future of PyPI](https://whoisnicoleharris.com/warehouse/) +- [Accessibility on Warehouse (PyPI) (blog, May 2018)](https://whoisnicoleharris.com/2018/05/17/warehouse-accessibility.html) +- [User Testing Warehouse (blog, Mar 2018)](https://whoisnicoleharris.com/2018/03/13/user-testing-warehouse.html) +- [Designing Warehouse - An Overview (blog, Dec 2015)](https://whoisnicoleharris.com/2015/12/31/designing-warehouse-an-overview.html) diff --git a/docs/html/ux-research-design/research-results/mental-models.md b/docs/html/ux-research-design/research-results/mental-models.md new file mode 100644 index 00000000000..0491586efab --- /dev/null +++ b/docs/html/ux-research-design/research-results/mental-models.md @@ -0,0 +1,70 @@ +# How Users Understand pip + +## Problem + +We want to understand how pip's users understand pip as a tool: what they think it is and what it does. + +[Skip to recommendations](#recommendations) + +## Research + +In order to capture participants mental models of pip and how package management works, we asked participants the following questions: + +- In your own words, explain what pip is +- In your own words, explain what happens when pip installs a software package +- In your own words, explain what a Python package dependency is + +When we talk about mental models, we talk about "deep" or "shallow" mental models. When a user has a deep mental models of something, their have a deep understanding with a lot of detail, shallow models are the opposite. + +In order to evaluate those mental models - do they match the reality of pip and package management - we worked with the maintainers to identify 1. pip's behaviours and activities (18 aspects), and 2. the aspects of package dependencies (13), and what a Python package dependency is (10). We then scored participants' answers against those. + +## Results + +The analysis focused on participants with between 2 and 10 years of Python experience. + +Over 90% of participants did not have a deep understanding of pip - with limited understanding of what pip is, what it does during the install process, and of package management in general. +However, while participants' understanding was low, only 4 participants had factually incorrect understandings of what pip is and does. + +Participants had a slightly deeper understanding of what happens during a pip install process. The most in depth answer included 7 of the 13 identified aspects. The median was 3. Answers focused on resolving dependencies, finding possible package names, downloading assets and installing the package. + +Participants' understanding of software dependencies was again shallow - the most in depth answer included 8 identified aspects. The median was 3. Answers focused on the fact that software dependencies were a result of code reuse, that constraining package versions reduced the possibility of dependency conflicts. + +The full data is available in[ this spreadsheet](https://docs.google.com/spreadsheets/d/1HBiNyehaILxhzZKWcBavkKXDzJr6gIt_Y8Jm8RRgJYg/edit#gid=0). + +### Responses to "In your own words, explain what pip is" + +> "pip is a standard command-line tool for managing python packages. It has three primary functions: (1) obtaining & caching python packages and/or their dependencies from a repository (typically pypi), (2) building (if needed) and installing python packages--and related dependencies--to a 'site-packages' location in the python path, and (optionally) (3) uninstalling previously-installed packages." **- participant 242608909 (Scientist, Professor in the Earth and Atmospheric Sciences Department, using Python for 7 - 10 years)** + +> "Pip is a package management system for python. Kind of like apt in linux, it can be used to install packages in public or private repositories into the current version or environment of Python that invoked the pip command." **- participant 240364032 (Professional software developer using Python for 7-10 years)** + +> "pip allows to install/update remove python libraries in your environment. pip manage the library. you will need something else to manage your environment. To use it the easiest is pip install `package-name` I recommend using a requirements.txt and add as you go the library and do pip install -r requirements.txt each time. it avoid to forget a library at the end of the project :)" **- participant 241178995 (Data scientist working in software engineering)** + +> "python's npm/cargo/opam... dedicated package manager and ecosystem for python libraries and applications" **- participant 240306262 (self-taught Python creative artist and web developer, using Python for 5-6 years)** + +> "A tool to download & install packages and resolve dependencies. I see it in the same area as yum, zypper or apt-get install in the Linux world." **- participant 240306204 (Using Python for scientific research and data analysis for 3 - 4 years)** + +> "Pip is the tool primarily used in the Python community to install packages. ("Package" means two different things in Python; it can be a target of the `import` statement that includes modules and other packages, or it can mean a collection of code with a defined interface that can be installed for reuse. I'm referring to the second thing here.) Pip's implementation defines what it means for a package to be installed in a Python environment. Any other tool that wishes to install software into a Python environment (e.g. conda) must match Pip's implementation." **- participant 240313922 (Computer security researcher at a university, using Python for 7-10 years)** + +### Responses to "In your own words, explain what happens when pip installs a software package" + +> "I think pip looks up package "tea" in the repository of packages (PyPI by default, but can be changed). If it doesn't find it, it gives an error. If it exists, it downloads some information about the package, like what form it exists in. This can be a wheel, or a package that needs to be built. If it is a wheel, it checks the dependencies and installs them, then it installs the wheel (not sure what this means, probably it extracts it). The wheel is specific to a python distribution and base OS, so it might be available on certain platforms but not others. If it is a package that needs to be built, pip downloads the package source (or clones the repository), and runs setup.py, which installs dependencies and other packages, then the package itself. I forgot to mention that before installing there is some check for checking compatibility of the version required and the versions required by other packages." **- participant 240426799 (Scientific researcher - data analysis and computer vision models, using Python for 5-6 years)** + +> "pip searches for a package source (and for me uses the default, so Pypi), then ask the package source for a package with the given name and versions (if specified), then if the package is available download the package in the most appropriate format (depending on my platform), then unzip the package and runs the installer (most probably calls setuptools with the included setup.py file) which will perform the required installation steps. This installation process may contain dependencies (typically specified in setup.py), which will trigger the same process for the dependencies, and so on until all dependencies are installed (if everything is OK)." **- participant 240670292 (Software developer industrial systems control, using Python for 5-6 years)** + +> "Pip checks PyPI (default package index, assuming that wasn't overridden) for the package matching `tea`. It uses the various specifiers (eg. OS compatibility, Python compatibility, etc) to find the latest version of `tea` compatible with my system. Within that version, it finds the best possible installation match (eg. a `wheel`, if supported on my system and my version of `pip` contains the relevant versioned support [eg. most recently manylinux2010], potentially falling back all the way to a source distribution). After downloading the relevant distribution, it performs the same operations recursively up the dependency chain as specified by the `install_requires` of the `setuptools.setup()` method. After grabbing all relevant packages, it performs the installations as specified in their setup methods -- generally, this involves extracting python files to specific system paths, but various levels of complexity may be added as need be such as compilations, system library bindings, etc. I believe the new resolver changes the above by performing all the lookups simultaneously (eg. by building and solving a dependency graph rather than traversing incrementally) but have not yet read the PEP to learn more. I've answered the above with setuptools in mind -- I believe there was a step added recently to check pyproject.toml first to allow for alternate systems here, but I find the added customization to be a net negative to the ecosystem and have not yet played with it -- the entire Poetry/Pipenv/Pipfile.lock/Flit thing just seems to be adding unnecessary complexity; users who know what they're doing have solved all these issues years ago for their packages and users who find the porcelain makes their lives easier are likely going to run into UX trouble no matter the veneer." **- participant 241463652 (Using Python for 5-6 years)** + +> "pip accesses the tea package from pypi (guessing that's where, online at least) and downloads a copy of the files into my local venv" **- participant 243434435 (Data analysis & machine learning, using Python for 1-2 years)** + +> "Looking up the latest version of of the package from pypi" **- participant 243897973 (Software testing/writing automated tests using Python 3 - 4 years)** + +> "Download, unpack, sometimes compile a module for my target arch" **- participant 243428875 (System administration using Python 7 - 10 years)** + +## Recommendations + +It's difficult to know what to recommend. Some ideas: + +- Question: Is it actually necessary for users to know everything that pip is doing? +- Better documentation: + - Describing the "blocks of functionality" that pip carries out and how to deal with them when it breaks + - Curating package manager training and help + - Improving pip output to expose the different pip functionality blocks diff --git a/docs/html/ux-research-design/research-results/override-conflicting-dependencies.md b/docs/html/ux-research-design/research-results/override-conflicting-dependencies.md new file mode 100644 index 00000000000..b087a8070b9 --- /dev/null +++ b/docs/html/ux-research-design/research-results/override-conflicting-dependencies.md @@ -0,0 +1,62 @@ +# Providing an override to install packages with conflicting dependencies + +## Problem + +Currently, when a user has dependency conflicts in their project they may be unaware there is a problem, because pip will install conflicting packages without raising an error. + +The new pip resolver is more strict and will no longer allow users to install packages that have conflicting dependencies. + +As a result, some users may feel that newer versions of pip are "broken" when pip refuses to install conflicting packages. + +For this reason, the pip team wanted to know if they should provide an override that allows users to install conflicting packages. + +[Skip to recommendations](#recommendations) + +## Research + +We published a survey with the following introduction: + +
+Imagine you have packages tea and coffee: + +tea 1.0.0 depends on water <1.12.
+coffee 1.0.0 depends on water>=1.12
+ +Installing tea 1.0.0 and coffee 1.0.0 will cause a conflict because they each rely on different versions of water - this is known as a "dependency conflict". + +The pip team has recently changed the way that pip resolves dependency conflicts. The new implementation is stricter than before: pip will no longer install packages where there is a dependency conflict - instead it will show an error. + +The purpose of this survey is to gather feedback on providing a way to override this behaviour. + +All questions are optional - please provide as much information as you can. + +
+ +We then asked users: + +- If pip should provide an override that allows users to install packages when there are dependency conflicts +- Why they answered yes or no +- For users that answered yes, we asked: + - When they would use the override + - How often they would use the override + - How easy it would be to find a workaround, if pip did not provide an override + - What syntax they prefer + +## Results + +In total, we received 415 responses to the survey. + +An overwhelming majority (>70%) of respondents indicated that they want some kind of override that allows them to install packages when there are dependency conflicts. Despite desiring this feature, most respondents said if it exists they would use it "not often" — this indicates that it is an advanced feature that is not critical to day-to-day usage. Nevertheless, because it would be difficult or very difficult to find a workaround (>60%), we suggest that pip should offer a override feature (see recommendations, below). + +Over half of the respondents said that `pip install tea coffee --ignore-conflicts` was the most ideal syntax for this command when installing multiple packages at once with a conflicting dependency. When using the `pip install —-ignore-conflicts` command, a majority (>48%) of respondents said they would prefer pip to install to the most recent version of the conflicted dependency. + +Most respondents suggested that installing the latest version by default is safer, because it could include security fixes or features that would be difficult to replicate on their own. They also trust that dependencies will be largely backwards-compatible. However, they said it was very important that it is necessary to have a way to override this default behavior, in case they need to use an older version of the conflicted package. + +## Recommendations + +Based on this research we recommend that the pip team: + +- Implement an `--ignore-conflicts` option, that allows users to install packages with conflicting dependencies +- Ensure that `--ignore-conflicts` installs the most recent version of the conflicting package. For example, for conflicting package `water<1.1.2` and `water≥1.1.2`, pip should prefer to install `water≥1.1.2`. +- Allow users to override this default behavior by specifying the version of the conflicting packages. For example, `pip install tea coffee water==1.1.1 --ignore-conflicts` +- Warn users that they used the `--ignore-conflicts` flag and that this may cause unexpected behavior in their program diff --git a/docs/html/ux-research-design/research-results/personas.md b/docs/html/ux-research-design/research-results/personas.md new file mode 100644 index 00000000000..06b84a8d4ef --- /dev/null +++ b/docs/html/ux-research-design/research-results/personas.md @@ -0,0 +1,250 @@ +# pip Personas + +## Problem + +We want to develop personas for pip's user to facilate faster user-centered decision making for the pip development team. + +[Skip to recommendations](#recommendations) + +## Research + +From early interviews with pip users, and from desk research into the different communities that use Python, it was our expectation that there were large communities who were not professional software developers. For example the SciPy library is widely used in the science and engineering communities for mathematical analysis, signal and image processing. + +Based on this, we expected a lot of these users would have different expectations, challenges and needs from pip. + +Our hypothesis was that: + +1. Python users fall into 3 main user types - a software user, a software maker and a software/package maintainer +2. That the majority (over 60%) would define themselves as Python software users +3. That the minority would define themselves as Python software maintainers + +### Usertype definitions + +During the research we've met different user types in the Python community. The 3 types of Python users, we proposed were: + +#### The Python Software User + +"I use Python software mainly as a tool to help me do what I want to do. This might be running scientific experiments, making music or analysing data with Python software I install with pip. I don't write Python software for others." + +#### The Python Software Maker + +"I use the Python software language and Python software packages to make software for others, mostly for other people. An example might be - building web applications for my customers. To make this web application I might use the Django framework, and a number of Python packages and libraries." + +#### The Python Package Maintainer + +"I spend a lot of my time creating Python software packages and libraries for other people to use in the software they make. I might make Python packages and libraries and then publish them on pypi.org or other software repositories." + +## Results + +During our research we found that these user types did fit with participants' sense of their usage of Python. Participants did not identify significantly different Python user types when asked. + +Each of these user types is a spectrum. Some Python users after time, and with experience/training, a need to use code more than once, started to make their own Python software. + +Identifying as one of these user types does not preclude users from also being another user type. Python users were more likely to Python software makers, but rarely Python software maintainers. + +Most (86%) participants identified as being a Python software user. This ranged a) from using Python applications - SciPy, Scikit-Learn - as a tool, with no knowledge, or interest to do more, to b) more advanced usage of Python involving modifying others code/scripts, possibly using libraries to create code specifically for their needs. + +75% identified as a Python software maker - as with Python software user, this ranged from writing basic scripts, code, to being a professional software developer. + +40% identified as a Python software maintainer - the activities of a maintainer were seen as only available to someone who had many years of Python experience, was heavily involved in a particular package or application, or did it as part of their job. + +### I am a Python software user + +As expected, almost all participants identified as a Python software user (86%). This was the most fundamental user type - both trained software developers and those who came to Python as a result of their job were users. + +Non-software developer users identified Python as a language to get stuff done - + +> "Almost everyone falls into the user (category) - that’s the target. It's not an obscure language that's meant for specific domains - it's a broad general purpose language designed to get stuff done. It's used by many who don't know other languages, they just need a language to get what they're doing finished." **- Participant 240312164** + +However, "using Python software" meant different things depending on who you ask - participants identified as a Python software user on a spectrum. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
I am a Python software userNumber of responses
I agree50
I disagree4
I have no opinion11
I strongly agree70
I strongly disagree4
Grand Total140
+ +![Pie chart with responses to question - I am a Python software user](https://i.imgur.com/ir3tP3B.png) + +#### Low end of the spectrum + +Python software applications were identified by some as a tool they use to do their "actual" work - the scientist, the data analyst, the engineer, the journalist. + +Here, they were "using" Python applications like SciPy, PsychPi, NumPy, to run scientific experiments, to gather data, to analyse data, with the objective of creating knowledge to make decisions, to identify what to do next. + +These were users who 1) who were new to Python software, 2) came across these Python applications in their profession, and used them as tools. + +They describe NumPy, or SciPy as a Python software application in itself, analogous to being a Windows user, or a Mac user. + +These users are not "classically trained programmers" as one participant identified themselves. As a result, they may not have the training, or knowledge about programming concepts like software dependencies. When they are expected to deal with complex or confusing error messages or instructions they have problems, often stopping them. + +#### High-end of the spectrum + +Python users who "move up the spectrum" to more advanced Python usage had been using Python for longer periods - many years. + +Again they may not have been classically trained developers, but through exposure - from work colleagues and their own usage - they started to experiment. This experimentation was in the form of modifying others scripts, taking classes, reading books so they could use code for other purposes. + +This was _making_ software - this software could be used by them as part of their day-job, but it could also be used by many others. + +We asked participants to explain the progression on this user spectrum - what is the difference between a user and a maker? + +Participants spoke about "are you working on something reusable or are you using the tools to achieve a one time task?" + +> "I didn't have classic software development training, more statistical analysis experience. I was clueless to the idea that it was a repository that anyone could upload packages to and become a maintainer." **- Participant \_240396891 (Data scientist at an applied research lab using Python do to network traffic analysis/parsing or Machine Learning)** + +> "Firstly I use my own software written in Python, I use Python libraries from pip. I use Django, Flask, libraries like requests." **- Participant 240302171** + +> "I am not a classically trained programmer, so it's a great way for me to learn and keep current in techniques. Not being a classically trained programmer, in some cases it detracts, I have a reasonable knowledge of the way to use hashes, but if I wanted to change Python's hash I'd have to read books. I can find information out there." **- Participant 240312164 (Nuclear physicist using Python for computer simulations, designing experimental methods)** + +### I am a Python software maker + +Being a "Python software maker" was a natural progression for some Python users, particularly those who had software development training - either on the job, personal learning or formal education. This training was important to understand fundamental programming concepts. + +As discussed earlier, some participants identified as "advanced" Python users, using Python software to modify or create other software. These users were likely to progress onto being software makers. + +55% of participants who identified as a software maker had between 5-20+ years of experience with Python. Only 18% of software makers had less than 2 years of experience. + +![Pie chart with responses to question - I am a Python software maker](https://i.imgur.com/aqg1kaL.png) + +We did not ask these participants about the "quality" of the software they created, but apart from the professional software developers, the opinion of these users was they were not software developers. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
I am a Python software userNumber of responses
I agree50
I disagree9
I have no opinion14
I strongly agree56
I strongly disagree10
Grand Total140
+ +Making software was as defined earlier as "are you working on something reusable or are you using the tools to achieve a one time task?" + +> "I'm using Python software and libraries to make this product I'm working on, it's foundation is based on Python, with React, D3 and all built on Python. The cloud assets are Python and testing is Python." **- Participant 240315927 (a professional IT developer building a Python based data analysis application)** + +> "I make software in Python. My day job is making software in python. Mainly Django web design. I work for a retail company, where I write calculating orders, creating data in other inventory management systems. Data analysis." **- Participant 240393825** + +> "I have written software, sometimes for business and personal reasons. At one point I worked on a django website project, that was being used by 1000s of people. I don't think any of my live projects are based. + +> "Most of it is for sysadmin, automation. I lke to use python instead of shell scripting. I manage a server with wordpress sites. I wrote a script to update these sites, mailman list and sql DB management, and for different utilities." **- Participant 240313542** + +> "I use Python for creating things - like outputs for data scientist, software engineer. I make software to look at patterns, and analyse stuff. I think I'm a maker because someone else is using - they are colleagues. Usually its non-technical colleagues. I produce outputs - make data understandable. They use the results, or a package it behind a flask app. Or analyse graphs." **- Participant 240426799** + +### I am a Python software maintainer + +The Python software/package maintainer user type was seen as requiring a significant amount of time and experience - domain experience as the software could be very specific (e.g. SciKit Learn, SciPy, etc), technical/coding experience, and experience in the community. You need to have spent time in doing the other jobs, before you could become a maintainer. + +For large projects it was seen as necessary to have core code contributors, and maintainers. Maintainers did not always write code - they could be more involved with technical architecture, technical design, than writing code. + +An aspect of the software maintainer role that wasn’t mentioned a lot was the community management aspect. + +![Pie chart with responses to question - I am a Python software maintainer](https://i.imgur.com/gXPc946.png) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
I am a Python package maintainerNumber of responses
I agree39
I disagree24
I have no opinion20
I strongly agree18
I strongly disagree38
Grand Total140
+ +> "You can become a maintainer once you get past a certain level of experience." **- Participant 240278297** + +> "To be a package maintainer, you'd have to spend a lot of time fixing issues, e.g. your package is on Github and you'd be looking at issues, reviewing PRs, writing documentation. A package maintainer is someone heavily involved in the project. They deal with more support calls, they do more thinking about issues to get your package into multiple environments. That's the good thing about the Python community - I was trying to use a Python package but there was an issue with the documentation. I said, there's a better way of doing this example. They answered and said "great, do you want to do it? Doing package maintaining, it doesn't interest me, I don't have time for it really - if I have a specific issue I will focus on it. It'd be nice (to do more)." **- Participant 240278297 (professional Python software developer)** + +> "I am a core developer of scikit-learn, I spend time writing code. These days strictly speaking - writing code is the least thing I do - mostly I do reviews of other people's code. There is a lot of API design work, it can translate into writing code. I may be the one writing the code or not. I am involved with the CI every now and then. [...] I have been the release manager for the last 2 releases. There are different types of maintainer - writing code maintainers, but you do need core devs writing code. But being a maintainer and building a community -that is about communication and PRs, and mentoring people." **- Participant 240306385 (core maintainer of SciKit-Learn)** + +## Recommendations + +### Provide documentation recommending "best/recommended ways" + +The majority of participants were using Python as a tool, as a participant said: "it's a broad general purpose language designed to get stuff done." + +The majority of participants - scientists, product/electronic engineers, data analysts, nuclear physicists - used Python for their work - they may write Python software, for themselves, possibly for colleagues. A smaller number are maintainers of widely used Python packages. + +As a result they are not classically trained software developers and so may not have "the right" understanding of important software programming concepts. + +Users of all types, and experience struggled with knowing the "right" way to do something. They often spoke about the "recommended way" to do something - to start a new project, to make a package: + +> "As a new comer, it's not easy to figure out what should be in the right way to structure a _setup.py_ or _pyproject.toml_. There is a good guide, but it's not easy to figure out what to use. I wish there was a guide like 'Make an application (or library) in 30 minutes'." diff --git a/docs/html/ux-research-design/research-results/pip-force-reinstall.md b/docs/html/ux-research-design/research-results/pip-force-reinstall.md new file mode 100644 index 00000000000..4ba84b876d3 --- /dev/null +++ b/docs/html/ux-research-design/research-results/pip-force-reinstall.md @@ -0,0 +1,102 @@ +# pip --force-reinstall + +## Problem + +Currently, when `pip install [package-name] --force-reinstall` is executed, instead of reinstalling the package at the version previously installed, pip installs the package at the newest version available. + +i.e. `pip install [package name] --force-reinstall` acts as `pip [package name] --upgrade` + +We want to find out if users understand (or desire) this implicit behaviour. + +More information can be found on [this GitHub issue](https://github.com/pypa/pip/issues/8238). + +[Skip to recommendations](#recommendations) + +## Research + +To help us understand what users want from the `--force-reinstall` option, we launched a survey with the following scenario: + +
+You have the requests package and its dependencies installed: + +requests==2.22.0
+asgiref==3.2.10
+certifi==2020.6.20
+chardet==3.0.4
+Django==3.1
+idna==2.8
+pytz==2020.1
+sqlparse==0.3.1
+urllib3==1.25.10
+ +You run 'pip install requests --force-reinstall'. What should happen? + +
+ +Respondents could choose from one of the following options: + +- pip reinstalls the same version of requests. pip does not reinstall request's dependencies. +- pip reinstalls requests and its dependencies, updating all these packages to the latest compatible versions +- pip reinstalls requests and its dependencies, keeping every package on the same version +- pip reinstalls requests, updating it to the latest version. pip updates request's dependencies where necessary to support the newer version. +- I don't know what pip should do +- I don't understand the question +- Other (allows respondent to provide their own answer) + +We also asked how useful `pip --force-reinstall` is, and how often it is used. + +## Results + +In total we received 190 responses to our survey, with 186 people telling us what pip should do when the `--force-reinstall` option is executed. + +![pie chart with survey results](https://i.imgur.com/yoN02o9.png) + +- **31.7%** (59/186) of respondents said that pip should reinstall requests and its dependencies, keeping every package on the same version +- **28%** (52/186) of respondents said that pip should reinstall requests, updating it to the latest version, with pip updating request's dependencies where necessary to support the newer version. +- **15.6%** (29/186) of respondents said that pip should reinstall requests and its dependencies, updating all these packages to the latest compatible versions +- **14%** (26/186) of respondents said that pip should reinstall the same version of requests, and not reinstall request's dependencies + +If we group responses into "upgrade" or "do not upgrade" (ignoring responses that could not be grouped), we find: + +- 46.32% (88/186) of respondents thought that pip should install the same version of requests - i.e. that `--force-reinstall` should _not_ implicitly upgrade +- 43.16% (82/186) of respondents thought that pip should upgrade requests to the latest version - i.e that `--force-reinstall` _should_ implicitly upgrade + +Most respondents use `--force-reinstall` "almost never" (65.6%): + +![screenshot of survey question of how often users use --force-reinstall](https://i.imgur.com/fjLQUPV.png) +![bar chart of how often users use --force-reinstall](https://i.imgur.com/Xe1XDkI.png) + +Amongst respondents who said they use `--force-resinstall` often or very often: + +- 54.54% (6/11) of respondents thought that pip should install the same version of requests - i.e. that `--force-reinstall` should _not_ implicitly upgrade +- 45.45% (5/11) of respondents thought that pip should upgrade requests to the latest version - i.e that `--force-reinstall` _should_ implicitly upgrade + +Respondents find `--force-reinstall` less useful than useful: + +![screenshot of survey question of how useful users find --force-reinstall](https://i.imgur.com/6cv4lFn.png) +![bar chart of how useful users find --force-reinstall](https://i.imgur.com/gMUBDBo.png) + +Amongst respondents who said they find `--force-resinstall` useful or very useful: + +- 38.46% (20/52) of respondents thought that pip should install the same version of requests - i.e. that `--force-reinstall` should _not_ implicitly upgrade +- 50% (26/52) of respondents thought that pip should upgrade requests to the latest version - i.e that `--force-reinstall` _should_ implicitly upgrade + +## Recommendations + +Given that this option is not regularly used and not strongly rated as useful, we recommend that the development team consider removing `--force-reinstall` _should they wish to reduce maintenance overhead_. + +In this case, we recommend showing the following message when a user tries to use `--force-reinstall`: + +> Error: the pip install --force-reinstall option no longer exists. Use pip uninstall then pip install to replace up-to-date packages, or pip install --upgrade to update your packages to the latest available versions. + +Should the pip development team wish to keep `--force-resintall`, we recommend maintaining the current (implicit upgrade) behaviour, as pip's users have not expressed a clear preference for a different behaviour. + +In this case, we recommend upgrading the [help text](https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-force-reinstall) to be more explicit: + +Old help text: + +> Reinstall all packages even if they are already up-to-date. + +New help text: + +> Reinstall package(s), and their dependencies, even if they are already up-to-date. Where package(s) are not up-to-date, upgrade these to the latest version (unless version specifiers are used). diff --git a/docs/html/ux-research-design/research-results/pip-search.md b/docs/html/ux-research-design/research-results/pip-search.md new file mode 100644 index 00000000000..641b30ca673 --- /dev/null +++ b/docs/html/ux-research-design/research-results/pip-search.md @@ -0,0 +1,145 @@ +# pip search + +## Problem + +By default, `pip search` searches packages on PyPI.org from the command line. However, the team are [considering removing it](https://github.com/pypa/pip/issues/5216), because they think it's not that useful and using too many resources on PyPI ([PyPI XMLRPC search has been disabled](https://status.python.org/incidents/grk0k7sz6zkp) because of abuse/overuse). + +[Skip to recommendations](#recommendations) + +## Research + +Prior to PyPI XMLRPC search being disabled, we: + +- Gathered feedback on pip search via the "buy a feature" survey +- Published a survey specifically about pip search, asking users about: + - Their current use of pip search + - How useful they find pip search results + - How clear they find pip search results + - Where users expect pip to search (e.g. PyPI vs private index) + - What data pip should search _other_ than project name + - What changes or additions they would make to pip search + +## Results + +In total, we received 1070 responses to the buy a feature survey, with 541 (50.4%) respondents selecting "Search pypi.org for packages" in their top 10 features. + +However, search ranked lower than the following features: + +1. Run pip without requiring any user input (e.g. in CI) _718_ +2. Show information about all installed packages _707_ +3. Show information about a single installed package _596_ + +We received 302 responses to the pip search survey, with 62 of the 302 (20.5%) respondents either not knowing that the command existed, never using it, or using it "rarely". + +We found that the remaining ~80% of respondents who do use pip search use it to: + +- Find/search for the right/new/alternate packages to install: + - Checking package name (verify correct spelling) + - Assessing functionality (check a package's description) + - Verifying availability (check if such package exists) +- Search for the latest version of a package (verify version) +- Find package libraries and new modules + +In general, pip search is regarded as: + +- more useful than not useful +- more clear than not clear + +When asked if pip should search on items _other_ than the package name, respondents most commonly asked to search the package description: + +![wordcloud of common search terms](https://i.imgur.com/lxS2TG6.png) + +Some users also mentioned that they would like the search to be configurable, e.g. by passing flags/options. + +When asked how they would improve pip search, users said they would improve: + +**1. Search methods:** + +- fuzzy search and insensitive case should be acceptable +- users should have the option to filter/sort by description, name, tag + +**2. Search results:** + +- relevancy: the results should show both the exact match and closest match +- order/category: the result should display items in a certain order, e.g highest number of downloads (popularity), development status (last updated/latest version), etc. +- there should be a limited number of search results + +**3. User interface:** + +- link package to pypi page +- use color coding / system for better clarity +- distinguish exact match search results from others: by highlighting, or using a different color +- indicate version compatibility + +## Recommendations + +### Deprecation strategy + +Given that the [PyPI](https://pypi.org/pypi) search API is currently disabled (as of 1st Jan, 2021) for technical and sustainability reasons, we recommend that the pip team display a clear error message to users who use the command: + +``` +The PyPI search API has been disabled due to unmanageable load. +To search PyPI, open your browser to search for packages at https://pypi.org +Alternatively, you can search a different index using the --index command. +``` + +In the longer term, **we recommend that the PyPI team investigate alternative methods of serving search results (e.g. via caching)** that would enable pip search to work again. This recommendation is supported by our research which suggests that many pip users find this functionality useful. + +If this is not possible, the pip team should create clear instructions that tells users what to use instead. Some suggestions (based on common user flows) are listed below: + +#### Finding a new package based on tags and keywords + +This is the most common feature that you would expect from `pip search` and likely the hardest to replace after deprecation. + +As mentioned above, the pip CLI should - as soon as possible - hide the full-trace error message present when a user types `pip search`. Instead, pip should show a message that encourages users to use the search index on the website itself (in their browser) by providing a link directly to [https://pypi.org](https://pypi.org). Also, pip should provide a short hint on how to use an alternative index. + +``` +$ pip search pytest + +The PyPI search API has been disabled due to unmanageable load. + +Please open your browser to search for packages at https://pypi.org + +Alternatively, you can use a different index using the --index command. + + pip search pytest --index +``` + +In addition, the pip team could implement an alternative to the PyPI search API that works without a hard dependency on a centralized service. Similar to other distribution systems like `apt` and `yum`, the metadata of all package names could be downloaded on the user's machine with an opt-in workflow: + +``` +$ pip search pytest +Using pip search on the command line requires you to download the index first. +Alternatively, you can open your browser to search for packages at https://pypi.org + +Download the index to /System/Library/Frameworks/Python.framework/ +Versions/2.7/Resources/Python.app/Contents/MacOS/search.db? (y/n) y +......... done! + + + +$ pip search pytest + +``` + +This is a more complex route that will require more engineering time, but can aim to provide command line users with a similar functionality to the old `pip search` command. It can also check the age of the local index and show a warning if it is getting old. + +#### Verifying the latest version of a package + +Users also use the `pip search` command to find or verify a particular package's version. + +As a replacement, the pip team could do either of the following: + +1. Extend the `pip show` feature to include known latest versions of the package; +2. Create a `pip outdated` command which scans the current dependency tree and outputs the packages that are outdated (compared to the latest versions on the configured index). + +### UX improvements + +Should it be possible to continue to support pip search, we strongly recommend the following UX improvements: + +- Adding support for [fuzzy search](https://en.wikipedia.org/wiki/Approximate_string_matching), or suggesting alternative/related search terms +- Adding support for case insensitive search +- Searching based on a package's description +- Linking search results to a package's PyPI page (where appropriate) + +Other user feedback (as detailed above) should also be considered by the team on a case-by-case basis. diff --git a/docs/html/ux-research-design/research-results/pip-upgrade-conflict.md b/docs/html/ux-research-design/research-results/pip-upgrade-conflict.md new file mode 100644 index 00000000000..6f35a550ca0 --- /dev/null +++ b/docs/html/ux-research-design/research-results/pip-upgrade-conflict.md @@ -0,0 +1,68 @@ +# pip Upgrade Conflict + +## Problem + +Currently, pip does _not_ take into account packages that are already installed when a user asks pip to upgrade a package. This can cause dependency conflicts for pip's users. + +[Skip to recommendations](#recommendations) + +## Research + +We published a [survey](https://bit.ly/2ZqJijr) asking users how they would solve the following scenario: + +
+Imagine you have package tea and coffee with the following dependencies: + +tea 1.0.0 - depends on water<1.12
+tea 2.0.0 - depends on water>=1.12
+coffee 1.0.0 - depends on water<1.12
+coffee 2.0.0 - depends on water>=1.12
+ +You have the following packages installed: + +tea 1.0.0
+coffee 1.0.0
+water 1.11.0
+ +You ask pip to upgrade tea. What should pip do? + +If pip upgrades tea to 2.0.0, water needs to be upgraded as well, creating a conflict with coffee... + +
+ +We gave users four choices: + +1. Upgrade tea and water. Show a warning explaining that coffee now has unsatisfied requirements. +2. Upgrade coffee automatically to 2.0.0 +3. Install nothing. Tell the user that everything is up-to-date (since the version of tea they have installed is the latest version without conflicts). +4. Install nothing. Show an error explaining that the upgrade would cause incompatibilities. + +We allowed users to post their own solution, and asked why they came to their decision. + +## Results + +In total, we received 693 responses, 407 of which included an explanation of why a particular solution was best. + +![](https://i.imgur.com/UdBWkaQ.png) + +- 497 responses (71.7%) preferred option 4: that pip should install nothing and raise an error message +- 102 responses (14.7%) preferred option 2: that pip should upgrade package_coffee +- 79 responses (11.4%) preferred option 1: that pip should upgrade tea and water +- 15 responses (2.2%) preferred option 3: that pip should install nothing and tell the user that everything is up to date + +From the 407 responses that answered "why" a particular solution was best, the following key themes emerged: + +- "explicit is better than implicit" - pip should not create "side effects" that the user does not understand, has not anticipated, and has not consented to +- pip should do everything in its power to avoid introducing conflicts (pip should not "break" the development environment) +- Telling the user that everything is up to date (option 3) is misleading / dishonest +- pip could be more flexible by: + - allowing the user to choose how they want to resolve the situation + - allowing the user to override the default behaviour (using flags) + +## Recommendations + +Based on the results of this research, the pip UX team has made the following recommendations to the development team: + +- While the current behaviour exists, [warn the user when conflicts are introduced](https://github.com/pypa/pip/issues/7744#issuecomment-717573440) +- [Change the current behaviour](https://github.com/pypa/pip/issues/9094), so that pip takes into account packages that are already installed when upgrading other packages. Show the user a warning when pip anticipates a depdenency conflict (as per option 4) +- Explore [the possibility of adding additional flags to the upgrade command](https://github.com/pypa/pip/issues/9095), to give users more control diff --git a/docs/html/ux-research-design/research-results/prioritizing-features.md b/docs/html/ux-research-design/research-results/prioritizing-features.md new file mode 100644 index 00000000000..12e6b8b244e --- /dev/null +++ b/docs/html/ux-research-design/research-results/prioritizing-features.md @@ -0,0 +1,156 @@ +# Prioritizing pip Features + +## Problem + +The pip development team is small, and has limited time and energy to work on issues reported via the [issue tracker](https://github.com/pypa/pip/issues). There is also a significant backlog of issues (782 as of November, 2020) for the team to respond to. +For the team to prioritize their work based on what will have the most impact, we need to develop a better understanding of what users want from pip. + +[Skip to recommendations](#recommendations) + +## Research + +To help answer this question, we developed a "buy a feature" survey, with the following scenario: + +
+Help us to understand what's important to you by participating in our "buy a feature" game: + +You have an allocated budget of $200 to spend on redesigning pip. + +With your $200 budget, "buy" the functionality you'd most like to keep. + +You don't have to spend the whole $200, but you should also not overspend your budget! + +
+ +We asked users to spend their first $100 on features related to `pip install`, and to spend their remaining $100 on other pip features. We also gave users an additional $10 to suggest a new feature: + +![survey question where users are asked to buy features for pip install](https://i.imgur.com/2QShgYo.png) + +![survey question where users are asked to buy features other than pip install](https://i.imgur.com/sY8gdXD.png) + +![survey question where users are asked to spend an additional ten dollars](https://i.imgur.com/hvgjdEG.png) + +## Results + +We received 1076 responses, 1070 of which were valid. The most popular features included the core competencies of pip: + +- Recreating an environment from a list of installed dependencies; +- Install, uninstall, and upgrade packages from a virtual control system, file, or local directory; +- Warn about broken or conflicting dependencies. + +### pip install + +The top ten features related to pip install were: + +![pip install results](https://i.imgur.com/1rNIOB7.png) + +1. Install and uninstall packages +2. Upgrade packages to the latest version +3. Warn about broken dependencies +4. Install a package from a version control system (e.g. Git, Mercurial, etc.) +5. Install packages as specified in a file +6. Install a package from a local directory +7. Verify downloaded packages against hashes +8. Install packages from an alternative package index, or indexes (default is PyPI only) +9. Install a package from wheels (no need for compiling code) +10. Control where you want your installed package to live on your computer + +### Other pip functionality + +The top ten features related to other pip functionality were: + +![other pip functionality results](https://i.imgur.com/xrp9XWw.png) + +1. Generate a list of installed packages that can be used to recreate the environment +2. Check that your installed packages do not have dependency conflicts +3. Run pip without requiring any user input (e.g. in CI) +4. Show information about all installed packages +5. Show information about a single installed package +6. Search pypi.org for packages +7. Show information about pip (version information, help information, etc.) +8. Download packages, build wheels and keep them in a directory for offline use +9. Manage pip's default configuration (e.g. by using configuration files) +10. Customise pip's output (e.g. reduce or increase verbosity, suppress colors, send output to a log) + +Results varied by the amount of Python experience the user had. + +
+See how likely users are to select a feature based on their experience level + +#### Verify downloaded packages against hashes + +![screenshot of verify downloaded packages against hashes](https://i.imgur.com/oVHOGBQ.png) + +#### Warn about broken dependencies + +![Screenshot of Warn about broken dependencies](https://i.imgur.com/uNv2tnG.png) + +#### Upgrade packages to the lastest version + +![Screenshot of Upgrade packages to the lastest version](https://i.imgur.com/pQgCLBO.png) + +#### Install packages from an alternative package index, or indexes + +![Screenshot of Install packages from an alternative package index, or indexes](https://i.imgur.com/E1LnTBt.png) + +#### Install packages as specified in a file + +![Screenshot of Install packages as specified in a file](https://i.imgur.com/87uh4xp.png) + +#### Install and uninstall packages + +![Screenshot of Install and uninstall packages](https://i.imgur.com/GRsazBy.png) + +#### Install packages from a version control system + +![Screenshot of Install packages from a version control system](https://i.imgur.com/iW7d0Sq.png) + +#### Install a package from wheels + +![Screenshot of Install a package from wheels](https://i.imgur.com/9DMBfNL.png) + +#### Install apackage from a local directory + +![Screenshot of Install apackage from a local directory](https://i.imgur.com/Jp95rak.png) + +#### Control where you want your installed package to live on your computer + +![Screenshot of Control where you want your installed package to live on your computer](https://i.imgur.com/32fpww2.png) + +
+ +## Recommendations + +### Environment recreation + +Environment recreation is already included in pip as part of the `requirements.txt` feature; however, with it's popularity and demand, we recommend that **pip should improve it's support of this feature.** + +- Improve environment recreation user output and help guides directly in the pip CLI; +- Improve pip documentation & user guide to prominently feature environment recreation as a core feature of pip; +- Improve environment recreation process itself by considering virtual environments as a core competency "built-in" to pip. + +**Recreating an environment from a list of installed dependencies was the most valued feature request overall** as well as in each user group, _except for those with less than 6 months of experience and those with 16-19 years of experience (for which it was the second most valued)._ + +When asked to enter a feature request with freetext, users placed the words 'built-in,' 'virtual,' 'automatic,' and 'isolation' alongside the word 'environment,' which suggest that users expect pip to recreate environments with a high level of intelligence and usability. + +**Selected direct quotes** + +> Make pip warn you when you are not in virtualenv + +> Automatic virtual env creation with a command line argument + +> Eliminate virtual environments. Just use ./python_modules/ like everyone else + +> I would love to see pip manage the python version and virtual env similar to the minicona + +> Would spend all my $200 on this: Integrate pipenv or venv into pip so installing an application doesn't install it's dependencies in the system package store. And allow pinning dependency versions for application packages (like how pip-compile does it) + +### Dependency management + +We recommend that the pip team improve warning and error messages related to dependencies (e.g., conflicts) with practical hints for resolution. This can be rolled out in multiple timescales, including: + +- Give hints to the user on how to resolve this issue directly alongside the error message; +- Prominently include virtual environment creation in the documentation, upon `pip install` conflict errors, and if possible as a built-in feature of pip; +- Upgrading the dependency resolver (in progress). + +It is clear that dependency management, including warning about conflicting packages and upgrades, is important for pip users. By helping users better manage their dependencies through virtual environments, pip can reduce the overall warnings and conflict messages that users encounter. diff --git a/docs/html/ux-research-design/research-results/users-and-security.md b/docs/html/ux-research-design/research-results/users-and-security.md new file mode 100644 index 00000000000..d9b79a057ff --- /dev/null +++ b/docs/html/ux-research-design/research-results/users-and-security.md @@ -0,0 +1,172 @@ +# How pip users think about security + +## Problem + +We wanted to understand how pip users think about security when installing packages with pip. + +[Skip to recommendations](#recommendations) + +## Research + +We asked participants about their behaviours and practices in terms of the security and integrity of the Python packages they install with pip, and of the software they create. + +We asked participants to tell us how often they: + +1. Carry out a code audit of the Python software they install with pip +2. Think about the security and integrity of the (Python) software they install (with pip) +3. Think about the security and integrity of the (Python) code they create + +## Results + +While the security and integrity of the software users install (51%) and make (71%) is important to research participants, less than 7% do code audits of the packages or code they install with pip. + +This is due to lack of time to audit large packages, lack of expertise, reliance on widely adopted Python packages, the expectation that pip automatically checks hashes, and reliance of the wider Python community to act as canary in the coalmine. + +This behaviour was common across all user types, and baselines of software development experience. + +These results - particularly the lack of expertise in auditing packages fits in with the overall findings that the majority of pip users are not "classically trained" (i.e. having formally learned software development) software developers and so lack the expertise and/or formal training in software development practices. + +There is a gulf between what the maintainers expect users to think, and worry about, and what the users actually worry and think about. Right now, pip leaves users to "fend for themselves" in terms of providing them with assurance of the software they install. This isn't meant as a criticism, but an observation. + +### Responses to question: before I install any Python software with pip, I carry out a code audit + +The vast majority of participants, 82%, do not (rarely or never) do a code audit of the software packages they install using pip, the reasons are explained below. + +| Before I install any Python software with pip, I carry out a code audit: | Number of responses | +| ------------------------------------------------------------------------ | ------------------- | +| Always | 3 | +| Frequently | 9 | +| Rarely | 66 | +| Never | 68 | +| I'm not sure what this means | 5 | +| No opinion | 13 | +| **Total number of participants** | **164** | + +### Responses to question: I think about the security and integrity of the software I install + +![screenshot of responses to question about security](https://i.imgur.com/wy4lGwJ.png) + +The vast majority of participants did think about the security and integrity of the software they installed - and unlike responses about code audits, in some cases participants made attempts to verify the security and integrity of the software they installed. + +Most attempts were made by those who had experience in software development, however in some cases, people gave up. + +Those who were not classically trained software developers did not know where to start. + +Both of these groups identified their "sphere of influence" and did their best to cover this. + +### User thoughts about security + +#### Responsibility as author + +Participants who spent a lot of their time writing Python code - either for community or as part of their job - expressed a responsibility to their users for the code they wrote - people who wrote code which was made public expressed a stronger responsibility. + +They thought about where the software would be used, who would use it, and possible attack surfaces. + +> "On the basic point, I have to think about attack surfaces. If I am writing the thing (software), I have to give a crap. I have to answer the emails! In the code I push to[ pypi.org](http://pypi.org/) I think about it doubley. What could people do with this code? Whether I do a good job, that's different! I am aware of it when publishing it or making it publically available. Whether I do a good job, that's different! I am aware of it when publishing it or making it publically available. I rely on community resources - Python security related, I follow security people blogs, Twitter. I use Hypothesis for fuzz-testing. I also rely on having security policies in place and a reporting mechanism. I steer clear of crypto, I rely on other peoples. There's a certain amount of knowledge in the Python community, I am actively involved in it. If something happens, I will hear about it. I use Twitter, if something happens, in the morning it can take me awhile to figure out what's happened. I have a lot of trust in the ecosystem to be self healing. As long as you don't stray too far-off the reservation (into using odd or uncommon or new packages), it's a better sense of security." **- Participant (data scientist turned Python developer)** + +> Yes, because I'm liable for that. If the problem is my code, and I deliver something and they get attacked. I'm screwed. **- Participant (professional Python developer and trainer)** + +#### Reliance on software packages + +Participants also explained they rely on code security scanning and checking software packages. + +> "I use linters (Bandit), I scan the code I have created and when there is an issue I raise a red flag." + +> "I use Hypothesis for fuzz-testing." + +#### Reliance on good software development practices + +A small number of participants e### Selected quotes from research participants +xplained they have good software practices in place, which help with writing secure software. + +> "We have a book about ethics of code - we have mandatory certification." + +> "I also rely on having security policies in place and a reporting mechanism. I steer clear of crypto, I rely on other peoples." + +Of the users who have used pip's hash checking functionality: + +- One finds the error messages "too annoying and loud", and has difficulty matching the file name to the hash +- Another finds the process of explicitly pinning hashes to be too tiresome (especially for dependencies) + +One user mentioned that he likes [NPM audit](https://docs.npmjs.com/cli/v6/commands/npm-audit) and would like to see something similar in the Python ecosystem. + +#### Lack of time + +The lack of time to carry out the audit of the package code, and that of the dependencies, was cited as a very common reason. In most cases participants used Python code as a means to achieving their goal. + +#### Lack of expertise to carry out the audit + +The lack of expertise or knowledge of auditing software was mainly due to participants expertise not being software development. However, in the case participants were "classically" software developers, lack of expertise was also a commonly given reason for not carrying out audits. + +#### Use of only widely used, well-established packages + +Use of well-established, high-quality packages was a common reason amongst all types of participants - professional Python software developers and those who used Python as a tool. + +"Well-established, high-quality packages" were defined by users as packages that: + +- have been in existence for many years +- are popular, or commonly used by those in their community or industry +- have responsive maintainers +- maintained by people the participant has heard of +- have many hundreds or thousands of users +- are in active development (many open issues, many forks, Github stars) +- are developed in the open, and transparently +- their history is known, or can be found out publicly + +#### Reliance on the Python community to find issues + +There was a reliance on the community to find issues and make them know publicly - "Many eyes shallow bugs". + +> "I rarely do code audits. Most of the time I rely on the opinions of the community. I look at how many maintainers there are. Maybe it's not good practice but I don't have time to go through the code." **- Participant 240315091** + +#### Use of only internal packages + +> "I only install internal packages, so I don't need to worry about this." + +This theme was not that common, mainly in large software development environments or where security was of high importance. + +#### Expectation that pip audits packages + +Some users expect/assume that pip (and PyPI) should "protect" them from malicious actors - e.g. by automatically checking hashes, or detecting malicious packages. + +> "If I was downloading a package on my own I check the hash, if it's installed by pip, then no. I expect pip to do it. If it doesn't do it, it does surprise me. Every package manager checks the hash against what it downloads. The hashes are already known on pypi." **- Participant 240312164 (Nuclear physicist)** + +#### Other notable comments + +> "Never. I should but I never do [audit code]. I don't stray, I am risk adverse. I install packages that are good already. I consider my risk surface small. I don't have time or resources to audit them. I have sufficient faith in the ecosystem to be self-auditing. If something turned up in a well known package, the community is well known for making a stink. And anyway a code audit wouldn't pick it up." **- Participant 240326752 (professional Python developer)** + +> "On the private level (work) the code is developed internally. I don't audit the code on pypi - due to lack of time auditing the dependencies, and I trust it. I know they had a security breach a few years ago, but it doesn't happen that often. I know they don't audit anything but I still don't audit the code." + +> "I wouldn't know how to [audit code], also I'm writing this stuff for myself. It'll work or not. Sometimes I end up installing 2 or 3 packages and find out that I need to install something else. I move on if it doesn't work. The last resort is I will write the code myself." + +> "I'm quite trusting - Python is open source, I'm assuming that if a package is on[ pypi.org](http://pypi.org/) - it must be alright. I install the package first, then I look at it. I find a package by figuring out - we need to do a certain task, we search for it on the Internet, look at the documentation, we install it and then see if it is what we want" **- Participant 240278297** + +> "If I want to install a package, it's for a reason. I want to calculate the azimuth and elevation of the moon with PyEphem. Do a code audit? Pffff. Most of the stuff I do is banal. It needs to meet a dependency, so I install it. I'm not going to do a code audit. I don't care. Never, but this is one of the things - is the package on pypi the exact source I see on Github? You could end up with files that are distributed differently. Probably (I don't do it) because I am too scared to look. There is this thing that pip verifies (the packages) hash - so that is a feature to guard against this. What is the hash of? No idea. It's located in the local python install." **- Participant 240426799 (systems administrator)** + +> "No [I don't audit code]. [laughs] Coz, I'm not going to read thousands of lines of code before I install a package. Oh my God. [..] I wouldn't be able to find it. I'm trading off - honestly how popular the package is, number of stars on GH. pypi doesn't have any UI way to tell me how many downloads it has. If it did I would use that." **- Participant 240386315 (IT administrator)** + +> "Well, I don't have the background to do a code audit of something like Numerical Python. Most packages I use are huge. Most people aren't doing code of those packages, except the maintainer. I am relying on whatever is built into pip to do package security. I also assume if there is an exploit someone will find it and let the world now. I'm really lazy." **- Participant 240312164 (Nuclear physicist)** + +> "I would like some security advisor, [like in npm](https://docs.npmjs.com/auditing-package-dependencies-for-security-vulnerabilities) - it works very well, when you install a package "there are security vulns. with this package - 1 low, 5 medium, 8 high. I haven't come across security issues with Python packages." **- CZI convening research participant** + +## Recommendations + +### Provide package security guidance or auditing mechanism + +A small number of participants (3-4) over the research period mentioned the[ NPM audit command](https://docs.npmjs.com/auditing-package-dependencies-for-security-vulnerabilities) as an example of a good way to assess package security. It may provide a model for how to approach this user need. + +### Automatically check package hashes + +pip should **by default** check packages hashes during install, providing a way for users to turn this behaviour off. + +In the case of no hash being available, pip should warn users and provide recommendations for users - from simplest to most advanced. + +### Mechanism to report suspicious packages + +Users should have a mechanism to report suspicious, or malicious, packages/behaviour. Where this mechanism should exist is open to discussion. The minimum should be a mechanism for users to flag packages on pypi.org. + +### Improve the output of pips activities easier to understand + +Right now pip's output is overwhelming and while it contains a lot of information, little of it is perceivable to the user - meaning is lost in "the wall of text". + +Pip's output must be redesigned to provide users with the right information - including security warnings - at the right time. diff --git a/docs/html/ux_research_design.rst b/docs/html/ux_research_design.rst deleted file mode 100644 index 165b6949670..00000000000 --- a/docs/html/ux_research_design.rst +++ /dev/null @@ -1,81 +0,0 @@ -==================== -UX Research & Design -==================== - -Over the course of 2020, the pip team has been working on improving pip's user -experience. - -Currently, our focus is on: - -1. `Understanding who uses pip`_ -2. `Understanding how pip compares to other package managers, and how pip supports other Python packaging tools`_ -3. `Understanding how pip's functionality is used, and how it could be improved`_ -4. `Understanding how pip's documentation is used, and how it could be improved`_ - -You can read the `overall plan`_ and the `mid-year update`_ to learn more about -our work. - -How to contribute ------------------ - -Participate in UX research -========================== - -It is important that we hear from pip users so that we can: - -- Understand how pip is currently used by the Python community -- Understand how pip users *need* pip to behave -- Understand how pip users *would like* pip to behave -- Understand pip's strengths and shortcomings -- Make useful design recommendations for improving pip - -If you are interested in participating in pip user research, please -`join pip's user panel`_. -You can `read more information about the user panel here`_. - -We are also looking for users to: - -- `Give us feedback about pip's new resolver`_ -- `Tell us how pip should handle conflicts with already installed packages when updating other packages`_ - -Report UX issues -================ - -If you believe that you have found a user experience bug in pip, or you have -ideas for how pip could be made better for all users, you please file an issue -on the `pip issue tracker`_. - -Work on UX issues -================= - -You can help improve pip's user experience by `working on UX issues`_. -Issues that are ideal for new contributors are marked with "good first issue". - -Test new features -================= - -You can help the team by testing new features as they are released to the -community. Currently, we are looking for users to -`test pip's new dependency resolver`_. - -Next steps ----------- - -In the coming months we will extend this documentation to include: - -1. Summaries of our user research, including recommendations for how to improve pip -2. Tools for the pip team to continue to practice user centered design (e.g. user personas, etc.) - -.. _Understanding who uses pip: https://github.com/pypa/pip/issues/8518 -.. _Understanding how pip compares to other package managers, and how pip supports other Python packaging tools: https://github.com/pypa/pip/issues/8515 -.. _Understanding how pip's functionality is used, and how it could be improved: https://github.com/pypa/pip/issues/8516 -.. _Understanding how pip's documentation is used, and how it could be improved: https://github.com/pypa/pip/issues/8517 -.. _overall plan: https://wiki.python.org/psf/Pip2020DonorFundedRoadmap -.. _mid-year update: http://pyfound.blogspot.com/2020/07/pip-team-midyear-report.html -.. _join pip's user panel: https://tools.simplysecure.org/survey/index.php?r=survey/index&sid=827389&lang=en -.. _read more information about the user panel here: https://bit.ly/pip-ux-studies -.. _Give us feedback about pip's new resolver: https://tools.simplysecure.org/survey/index.php?r=survey/index&sid=989272&lang=en -.. _Tell us how pip should handle conflicts with already installed packages when updating other packages: https://docs.google.com/forms/d/1KtejgZnK-6NPTmAJ-7aWox4iktcezQauW-Mh3gbnydQ/edit -.. _pip issue tracker: https://github.com/pypa/pip/issues/new -.. _working on UX issues: https://github.com/pypa/pip/issues?q=is%3Aissue+is%3Aopen+label%3A%22K%3A+UX%22 -.. _test pip's new dependency resolver: https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020 diff --git a/news/10745.doc.rst b/news/10745.doc.rst new file mode 100644 index 00000000000..4094f41405c --- /dev/null +++ b/news/10745.doc.rst @@ -0,0 +1 @@ +Start using Rich for presenting error messages in a consistent format.