[GSOC 2020] Project ideas

Update: See #10099 for GSoC 2021 project ideas.

Here are some ideas for larger mypy-related projects for contributors who want to tackle something fairly big (but also with a big potential impact).

Deep editor integrations

Currently it's possible to run mypy daemon from an editor and display the list of errors within the editor, but we could go much further. Possible ideas include going to the definition of an arbitrary reference (such as a method, variable, type, etc.), and displaying the inferred type of an expression. IDEs such as PyCharm can do some of this already, but mypy could support these features more reliably in some cases, since it maintains a very detailed representation of the program internally. Also, this could be very helpful with editors that have no or limited built-in support for these features.

Better decorator support

Mypy can't properly support decorators that change the return type of the decorated function, such as contextmanager (contextmanager is special-cased using a plugin, but this approach doesn't generalize to arbitrary functions). Add support for PEP 612 draft to make this better.

Generalize mypyc IR to allow non-C backends

Currently the IR of mypyc, the compiler we use to compile mypy, is tightly bound to C. This makes it impractical to experiment with alternative backends, such as an LLVM back end or a completely custom back end that directly generates assembly.

Related issue: mypyc/mypyc#709

Faster callables and nested functions in mypyc

Currently calling nested functions and variables with a callable type is pretty slow in compiled code. These limitations reduce the usefulness of mypyc significantly, especially when compiling code that wasn't originally written with mypyc in mind.

Related issues: mypyc/mypyc#713, mypyc/mypyc#712 (both of these would be implemented in a GSoC project)

NumPy support

This is a big topic, but this can be approached a feature at a time. One of the main missing things is "shape types" -- there needs to be a way to express the number of dimensions in an array, at the very least.

I'd be interested in adding support for PEP 612. I'll take a look at the pyre implementation this weekend, re-read the PEP, and scope out a plan.

@hauntsaninja

@hauntsaninja Great! If you have any questions, I'm happy to help.

Another idea:

Detecting potentially undefined or misspelled locals

Some uses of undefined variables are not caught by mypy:

def f() -> None:
    if foo():
        x = 0
    print(x)  # No error

It would be useful to catch these (#2400). A related issue is reporting locals that are never read, as this is often an error (#76).

GSOC 2020 organizations are announced and mypy is one of them! see https://summerofcode.withgoogle.com/organizations/6527008550420480/

I am interested in Generalize mypyc IR to allow non-C backends and I will manage to have a plan by the end of February.

Hi!
I'd be interested in working in detecting potentially undefined or misspelled locals.
I will work out a plan and get in contact soon...

Hi!!
I am Sarvesh, I am interested in Faster callables and nested functions in mypyc and Numpy support.
I am working on Faster Callables and will get back soon with some solutuon.

Hello,
My name is Jonathan, I am actually focus on data science and I am really interested to work in Numpy Support. I once questioned regarding the unavailable of "shape types" . By working this project, will help my Data Science works too.

I am going to read the documentations and try to find out some insights which features can be improve for numpy. Thanks

Cheers,
Jonathan

@edwinjon

Hi, @edwinjon and @dubesar, glad to see someone interested in working on the mypy-Numpy topic!

As far as I know, there's a similar work that may help you develop your ideas and plans, check https://github.com/numpy/numpy-stubs.

@JukkaL

@JukkaL By numpy support I understand that the codes written in standard format needs to be changed to numpy format as for example changing arrays to numpy arrays.
Also by shapes you mean the shapes of the arrays?

Please elaborate that part!!

Also the Faster Callables part I checked the running time of the code and have addressed it in the issue. Please guide me further what has to be done in that, so that I can move in further with the work.

@JukkaL

Hi @JukkaL my name is Eslam Genedy , I am at third year in University ASU Computer Engineering Department , it is my pleasure to apply for mypy . I have a good knowledge in oop in python and numpy . finally I hope to be contributed in this project thanks in advance

@ilevkivskyi

To support NumPy, we'd need some form of "shape type" support. We want to specify the number of dimensions and the item type of an array, at least. Even better, it would often be useful to specify the exact size of an array, but this will be much harder to implement.

This example is from @ilevkivskyi's presentation from the Typing Summit at PyCon 2019:

from typing import Shape, IntVar, TypeVar

N = IntVar('N')
T = TypeVar('T')

def diff(a: ndarray[T, Shape[N]]) -> ndarray[T, Shape[N - 1]]:
    ...
def sum2d(a: ndarray[T, Shape[:, :]) -> ndarray[T, Shape[:]]:
    ...

This means that diff takes a one-dimensional array of size N and item type T, and it returns a one-dimensional array of size N - 1 and item type T. sum2d, on the other hand, takes an arbitrary-sized two-dimensional array with item type T, and it returns a one-dimensional, arbitrary-sized array with item type T.

A reasonable goal for GSoC would be to support specifying the item type and the number of dimensions (e.g. ndarray[float, Shape[:, :]] for a two-dimensional array). This would involve at least these steps (I'm leaving a lot of details out):

Add support for the new shape type syntax.
Implement basic type operations involving shape types, such as displaying shapes in error messages and subtyping.
Implement simple stubs for NumPy that use the shape types.
If needed, add a mypy plugin to handle some common NumPy operations that can't be supported via existing type system features.

This is quite a challenging project and requires a deep understanding of mypy type checking internals, so it's really only well-suited for somebody who has substantial experience with working on mypy (or another type checker), or perhaps has completed a course on type theory (beyond a compilers course).

@JukkaL

@JukkaL , I'd like to integrate PEP 612 and take the better decorator support

@joybhallaa That would be a really useful project! I'd suggest starting by closing a few smaller issues first to get familiar with working on mypy. (The same advice applies to anybody who's interested in diving into a major project. It's best to gradually learn the codebase. Otherwise these's a big risk that there's too much to learn at once, and you'll get discouraged.)

Since PEP 612 involves type inference and type variables, tackling one (or some) of these issues could be a good option: https://github.com/python/mypy/issues?q=is%3Aopen+is%3Aissue+label%3Atopic-type-variables

If you find a promising issue but you are not sure where to start, you may want to ask for hints in the issue.

@JukkaL

Since PEP 612 involves type inference and type variables, tackling one (or some) of these issues could be a good option: https://github.com/python/mypy/issues?q=is%3Aopen+is%3Aissue+label%3Atopic-type-variables

@JukkaL I've taken up an issue and after I'm finished with that I'll take up issues that you have mentioned. 👍

@JukkaL

Hi @JukkaL I am interested in contributing for adding better decorator support, RIght now I am working on the issues , is there anything which should be done parallely with this and are there any potential mentors for this project idea?

Clarification: The Faster callables and nested functions in mypyc GSoC project includes implementing both of the linked issues.

Note that Detecting potentially undefined or misspelled locals feels too small for a full GSoC project. If somebody wants to work on this topic in the context of GSoC, we can expand the scope by also detecting (some) uninitialized attributes, and preventing unsafe attribute deletions. I can provide more context if there is interest.

Hello ,

Can you give a bit of details for numpy support ?

@shadaabghani1

@shadaabghani1 It would be much easier me to help you, if you can ask more specific questions, as the topic is pretty wide ranging. I already gave an overview in this comment: #8373 (comment)

@JukkaL

@JukkaL I don't know if for the numpy support point you could be interested in this thread: https://llvm.discourse.group/t/numpy-scipy-op-set/768

Currently I'm writing sphinx docs like this and I wish there was a sphinx plugin or something that would ask mypy what the type of each attribute is:

    .. attribute:: callback
        :type: Callable[[], None]

        Blah blah. Boring text goes here.

@JukkaL

Hey @JukkaL, will mypy participate in GSoC this summer? I am interested in working on the editor integration of mypy.

@JukkaL

Hey @JukkaL, will mypy participate in GSoC this summer? I am interested in working on the editor integration of mypy.

Hi, since it may take Jukka several hours before he gets back to you, I will try to share what I know. Mypy is still deciding whether it will participate in GSoC 2021 but we are looking forward to it. For project ideas, once core team members have decided to participate, they will come up with some possible ideas or even post a public issue for brainstorming. The final ideas will most likely be summarized in a new issue. Anyway, please stay tuned to this issue and also the Gitter channel.

Hi, since it may take Jukka several hours before he gets back to you, I will try to share what I know.

Thanks for replying to this. I will add myself to the Gitter channel and try working on any easy to fix issue to get started.

Thanks for replying to this. I will add myself to the Gitter channel and try working on any easy to fix issue to get started.

Cool, apart from mypy's issue tracker, you may also find some easy-to-do issues at: https://github.com/mypyc/mypyc/issues. And for mypyc issues, feel free to ping me for a review

I am going to close this since I don't think there is anything useful to track here, the issues brought up can be tracked in their respective issues.

Navigation Menu

Repository navigation

[GSOC 2020] Project ideas #8373

Deep editor integrations

Better decorator support

Generalize mypyc IR to allow non-C backends

Faster callables and nested functions in mypyc

NumPy support

Detecting potentially undefined or misspelled locals

7 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[GSOC 2020] Project ideas #8373

Description

Deep editor integrations

Better decorator support

Generalize mypyc IR to allow non-C backends

Faster callables and nested functions in mypyc

NumPy support

Activity

hauntsaninja commented on Feb 12, 2020

JukkaL commented on Feb 12, 2020

JukkaL commented on Feb 20, 2020

Detecting potentially undefined or misspelled locals

TH3CHARLie commented on Feb 20, 2020

vpeiter commented on Feb 22, 2020

sPredictorX1708 commented on Feb 22, 2020

edwinjon commented on Feb 22, 2020

TH3CHARLie commented on Feb 22, 2020

sPredictorX1708 commented on Feb 22, 2020

egenedy97 commented on Feb 24, 2020

JukkaL commented on Feb 26, 2020

joybh98 commented on Feb 27, 2020

JukkaL commented on Feb 28, 2020

joybh98 commented on Feb 29, 2020

pr4k commented on Feb 29, 2020

7 remaining items

JukkaL commented on Mar 24, 2020

JukkaL commented on Mar 24, 2020

shadaabghani1 commented on Mar 30, 2020

JukkaL commented on Mar 30, 2020

bhack commented on Apr 2, 2020

Akuli commented on Aug 3, 2020

friyaz commented on Feb 3, 2021

TH3CHARLie commented on Feb 3, 2021

friyaz commented on Feb 3, 2021

TH3CHARLie commented on Feb 3, 2021

emmatyping commented on May 19, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions