General discussion about support for other languages than just Python

Is it possible to make jupyter support other language like julia and R?
Or at least provide some interface for other extension to integrate.

Clouds (@clouds56)

Clouds (@clouds56), at the moment no.

This would be a good spot to have people vote on other languages. Which ones do you want besides python?

Julia?

R?

Haskell?

C#/F#?

Scala?

Bash ?

SAS?

cling?

I don't see any point in voting on languages here — supporting two languages should be the same amount of work as supporting all languages. The whole point of Jupyter is that it defines a language-independent protocol … if there is a general vscode extension for notebooks, it should work with any notebook in any language (i.e. it should be decoupled from vscode-python, analogous to nteract/hydrogen).

That is, when you open a Jupyter notebook file (.ipynb), the kernelspec.name metadata should tell you the name foo of the kernel to run. You then look for a kernelspec file foo/kernel.json in the standard locations. This file looks something like

{
  "display_name": "Julia 1.2.0",
  "argv": [
    "/Applications/Julia-1.2.app/Contents/Resources/julia/bin/julia",
    "-i",
    "--startup-file=yes",
    "--color=yes",
    "--project=@.",
    "/Users/me/.julia/packages/IJulia/src/kernel.jl",
    "{connection_file}"
  ],
  "language": "julia",
  "env": {},
  "interrupt_mode": "signal"
}

and tells you exactly how to launch the kernel (via the argv). You create a connection file, pass this for the {connection_file} argument when starting the kernel, and then talk to the kernel over ZMQ in exactly the same way that you talk to the Python kernel.

hochshi (@hochshi)

Unfortunately supporting a kernel is not the same as supporting a language. We run a bunch of stuff that is specific to python in order to get things like theming, variables, startup directories, etc correct. None of those work without figuring out a language neutral or per language way to do them.

We will be supporting other kernels long before we support other languages. This item here:
https://github.com/microsoft/vscode-python/issues/3123
has the beginnings of how this will work.

hochshi (@hochshi) has implemented picking a kernel in remote situations here (which will hopefully go in soon)
microsoft/vscode-python#7790

Supporting just the the script itself, as an interactive notebook, without the Jupyther dependency would be even better.

Supporting just the the script itself, as an interactive notebook, without the Jupyther dependency would be even better.

@alexeypetrushin can you elaborate more? Jupyter is just a framework for hosting a process that's running your code. I don't think we'd reinvent this ourselves. Is there something you don't like about having jupyter installed?

You don't need a dependency on the Jupyter software stack if you simply speak the Jupyter messaging protocol to the kernel. e.g. nteract does this.

(You obviously need the software installed for any kernel you want to use, e.g. you need the ipython kernel installed if you want to talk to Python, which pulls in components from Jupyter. But you wouldn't need e.g. the jupyterlab component. And other languages might not need Jupyter software at all. For example, the Jupyter kernel for Julia has no dependency on the Jupyter software stack — it simply speaks the Jupyter protocol over ZMQ to anything that launches it.)

Unfortunately supporting a kernel is not the same as supporting a language. We run a bunch of stuff that is specific to python in order to get things like theming, variables, startup directories, etc correct. None of those work without figuring out a language neutral or per language way to do them.

By "theming" I guess you mean syntax highlighting etcetera? Why can't you simply read the language field of the kernelspec file and activate the syntax mode for the corresponding language?

Not sure what you mean by "startup directories" … the kernelspec file also tells you exactly how to launch a kernel, including any relevant environment variables.

I'm not speaking theoretically — JupyterLab and nteract do this in a language-neutral way already.

David Anthoff (@davidanthoff)

It's not the query that is failing. I can get the list. It's the inspect request.

For example, with this code here:

a = 5

The regex I was using returns me:

"Base",
"Core"
"Main",
"a"

All 4 of those give the same inspect result.

Trying with David Anthoff (@davidanthoff)'s query and regex gives the same result.

Should an inspect request for "a" throw a bounds error? Do you have a way for me to try the same inspect request in a cell (IPython does this if you prefix code with a ?, like ?x will inspect x)

I see. I am not sure if it is relevant, but these guys told me the following:

julia> a = 5
5

julia> varinfo(Main, r"a")
  name                    size summary
  –––––––––––––––– ––––––––––– –––––––
  Base                         Module
  InteractiveUtils 198.160 KiB Module
  Main                         Module
  a                    8 bytes Int64
  ans                  8 bytes Int64

julia> varinfo(Main, r"^a$")
  name    size summary
  –––– ––––––– –––––––
  a    8 bytes Int64
``
So if `varinfo` is used behind the hood, maybe you need to put pass in the `^` and `$` around the string?

That gives the same bounds error on the inspect request. Inspecting "^a$" instead of "a" gives the same problem.

Is the inspect request just sending some Julia code to eval in the kernel? What exactly is being sent? Or is there a special inspect Jupyter message?

Steven G. Johnson (@stevengj)

Wait, that may not be it. See that https://github.com/JuliaLang/IJulia.jl/blob/9b10fa9b879574bbf720f5285029e07758e50a5e/src/handlers.jl#L192 is called from the inspector implementaiton: https://github.com/JuliaLang/IJulia.jl/blob/9b10fa9b879574bbf720f5285029e07758e50a5e/src/handlers.jl#L236

Note that

julia> a = 5
5

julia> using IJulia

julia> IJulia.docdict("a")
Dict{String,Union{String, JSON.Writer.JSONText}} with 3 entries:
  "text/plain"    => "  No documentation found.\n\n  \e[36ma\e[39m is of type \e[36mInt64\e[39m.\n\n\e[1m  Summary\e[22…
  "text/markdown" => "No documentation found.\n\n`a` is of type `Int64`.\n\n# Summary\n\n```\nprimitive type Int64 <: S…
  "text/latex"    => "No documentation found.\n\n\\texttt{a} is of type \\texttt{Int64}.\n\n\\section{Summary}\n\\begin…

Is that the text that would be parsed by the

vscode-jupyter/src/client/datascience/jupyter/kernelVariables.ts

Lines 438 to 443 in 69a2fb5

    
           const type = TypeRegex.exec(text); 
        
           const value = ValueRegex.exec(text); 
        
           const stringForm = StringFormRegex.exec(text); 
        
           const docString = DocStringRegex.exec(text); 
        
           const count = CountRegex.exec(text); 
        
           const shape = ShapeRegex.exec(text);

But I think we are now in the guts where Steven G. Johnson (@stevengj) might be able to point some of this stuff out.

Yes this text here:

e[36ma\e[39m is of type \e[36mInt64\e[39m.\n\n\e[1m  Summary\e[

is the type of response we're expecting.

Is the inspect request just sending some Julia code to eval in the kernel? What exactly is being sent? Or is there a special inspect Jupyter message?

The inspect request is a special jupyter kernel message.

The docs for it are here:
https://jupyter-client.readthedocs.io/en/stable/messaging.html#introspection

We send the inspect_request with the 'code' value set to the name of the variable.

What are you sending as the cursor_pos in the inspect message? The stack trace from your screenshot to me looks like a string indexing error inside get_token at https://github.com/JuliaLang/IJulia.jl/blob/9b10fa9b879574bbf720f5285029e07758e50a5e/src/handlers.jl#L195 from your screenshot.

Is there any chance you could let me know the line number of the stack trace item that I marked with a red error from your screenshot?

Namely the line number of the stack trace in get_token.

We set cursor_pos to zero.

Trace for the get_token is at line 217:

" [4] get_token(::String, ::Int64) at C:\Users\Rich.julia\packages\IJulia\IDNmS\src\handlers.jl:217"

Closing this as other languages are supported in the new UI (with other extensions supporting their own kernels and language servers)

Navigation Menu

Repository navigation

General discussion about support for other languages than just Python #1536

71 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	const type = TypeRegex.exec(text);
	const value = ValueRegex.exec(text);
	const stringForm = StringFormRegex.exec(text);
	const docString = DocStringRegex.exec(text);
	const count = CountRegex.exec(text);
	const shape = ShapeRegex.exec(text);

General discussion about support for other languages than just Python #1536

Description

Activity

rchiodo commented on Apr 2, 2019

rchiodo commented on Jul 2, 2019

rchiodo commented on Jul 2, 2019

rchiodo commented on Jul 2, 2019

rchiodo commented on Jul 2, 2019

rchiodo commented on Jul 2, 2019

rchiodo commented on Jul 9, 2019

raghavgautam commented on Aug 23, 2019

chrishales709 commented on Sep 23, 2019

dr-br commented on Oct 9, 2019

stevengj commented on Oct 11, 2019

rchiodo commented on Oct 11, 2019

al6x commented on Nov 27, 2019

rchiodo commented on Nov 27, 2019

stevengj commented on Nov 27, 2019

71 remaining items

rchiodo commented on Jan 12, 2021

jlperla commented on Jan 12, 2021

rchiodo commented on Jan 12, 2021

davidanthoff commented on Jan 12, 2021

jlperla commented on Jan 12, 2021

rchiodo commented on Jan 12, 2021

rchiodo commented on Jan 12, 2021

davidanthoff commented on Jan 12, 2021

rchiodo commented on Jan 12, 2021

rchiodo commented on Aug 30, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions