Repository navigation
A *plea* to include binary API required by socket.io into NodeJS core. #8005
Description
The Problem
There is unending pain on Windows for anyone attempting to use the beloved socket.io library. This is rooted in the fact that you're forced to perform a compile and worse in a SDK (Visual Studio 2010) most Windows devs ditched some years ago. You can kinda/maybe/possibly hack around it if you have admin privs and a lots of Window experience but basically any app that dares to use this wonderful lib is in for a ton of support issues and pain and most end users won't be able to get it working.
Just google search for "win7 nodejs socket.io" and feel the pain out there.
Today I'm hitting this issue yet again as I'm trying to use a new PhoneGap component that would be very cool with remote logging if not for this ten pound gorilla standing in my path.
It's not just Windows that experiences this pain. Other platforms also suffer but for me today it's another Windows issue and this is a recurring nightmare for anyone doing node on Windows.
The Solution
IMHO I think the right solution is to remove the need for compiling for what I think is now a frequently used CORE feature of node by adding the API required by socket.io into the node core code. So the fix then just just a Node upgrade rather than a two day torture test.
The Plea
Socket.io is FAB! It opens tons of cool trick doors for node. Please remove the pain. All the node community will rejoice.
Activity
tohagan commentedon Jul 27, 2014
FYI ... I'm using node 0.10.29
TooTallNate commentedon Jul 28, 2014
socket.io's only native dependency is
ws, which actually has an optional compilation step that falls back to a vanilla JavaScript implementation if compilation fails.That said, the compiled portion of socket.io is optional and only benefits speed in the end. Where are you getting hung up exactly?
bnoordhuis commentedon Jul 28, 2014
I think what @tohagan is complaining about is that gyp / node-gyp defaults to VS 2010 when auto-detection fails and GYP_MSVS_VERSION isn't set in the environment. (Upstream gyp defaults to 2012 now, by the way.)
@tohagan You can set e.g. GYP_MSVS_VERSION=2012 or GYP_MSVS_VERSION=2013e (the 'e' stands for 'express edition') to pick a particular version of VS. See https://github.com/joyent/node/blob/v0.10.29/tools/gyp/pylib/gyp/MSVSVersion.py#L209-294 for the full list.
TooTallNate commentedon Jul 28, 2014
I'll look into updating gyp in node-gyp ASAP.
tohagan commentedon Jul 29, 2014
Yes .. As a developer with many years experience I can probably dodge this bullet and follow instructions on stackoverflow or here but the fact remains is that large numbers of end users of node apps and libs cannot. Yes I'm aware that
wsis the problem. So I'm pleading that you include it into CORE so we can avoid the compiler issue and keep sockets fast.There are tons of great node libs and apps that are simply very painful to deploy on Windows because of this issue and this pain has now subsisted for many months. Hacking the environment does not fix it for non tech users. Please realise that Node is now maturing and that it need to be "turn key" installed for all users - not just developers. That's the very cool dream of npm.
wsis a wonderful and essential part of NodeJS but because of the serious and constant pain this module creates for Windows deployment, I'm lobbying that it become part of NodeJS CORE library so we can finally bury the need to be compiling anything.So .. I'm pleading ... Can you please team up with the
wsteam so we can all rest easy at night knowing we won't have to face another request to install VS 2010 or any other developer tool in the morning.The fact is that now many NodeJS end users who install our node apps ...
(a) Don't have admin privs on Windows
(b) Don't have the technical expertise to dodge the
wsbullet.(c) Are Not developers
(d) Are Not Visual Studio developers
(e) Ditched Visual Studio 2010 long ago
However, these end users typically can download and install the next release of NodeJS. So I think
wsand NodeJS CORE belong together - precompiled for each platform so we can forget this nightmare.Just do a Google search for "win7 nodejs socket.io" and you'll get the idea.
mwbrooks commentedon Jul 29, 2014
+1 Win7 support for socket.io would be incredibly helpful. Thanks for opening this discussion @tohagan!
bnoordhuis commentedon Jul 29, 2014
You know, incorporating ws kind of makes sense in a twisted way. Websockets are a common enough use case that it makes a reasonable core addition.
TooTallNate commentedon Jul 29, 2014
None of this addresses the fact that
wsfalls back to JavaScript implementations of the modules, and thus native compilation is not a requirement.So I ask again, what is really the issue? Where are you being prevented from using socket.io?
If you do
npm install socket.io, and the native compilation part fails, then socket.io should still be installed and doingrequire('socket.io')in a script at that point should still work. Is this not the case?tohagan commentedon Jul 29, 2014
Thanks @TooTallNate ... I will retest this. If it does work then perhaps the failure message on install need to switch to being a warning somehow (not simple I suspect).
Another longer term solution for Node might be to ship the GNU gcc compiler as part of Windows NodeJS so we get a more uniform compiling environment that no one needs to install separately. Perhaps it becomes part of gyp/ node-gyp. I realise that for Windows coders, VS is much better supported.
Might even be able to auto download / install mingw's port of gcc as part of the install process. Not sure if their licence will allow this. Might need to chat with them first.
http://www.mingw.org/
IMHO I still think
wsbelongs in CORE even with this.Qard commentedon Jul 30, 2014
+1 on core support for websockets.
On a side note, including ws in core doesn't solve the native compilation issue for any of the other many native modules out there. The proposed solution does not solve the actual problem, which is that compiling native modules on Windows just sucks. With Google switching to gn, the future of gyp is a bit unknown--perhaps the state of native modules on Windows will improve as we work through the deprecation of gyp.
tohagan commentedon Aug 1, 2014
I managed to get past the issue for myself by setting GYP_MSVS_VERSION to 2012. Thanks!
rlidwka commentedon Aug 6, 2014
Why do you even have to use it on native Windows? Just develop it using vagrant or something like that.
dougwilson commentedon Aug 6, 2014
FWIW, this is what the Perl community does for Perl on Windows (Strawberry Perl distribution).
tohagan commentedon Aug 7, 2014
@dougwilson Very interesting. At present of course, the NodeJS community has invested some effort in Visual Studio C++ code & VS.NET projects so I guess if they went down this path you don't just drop the support for this existing code but rather, offer a migration path ... so there may be a period of time where you support both OLD (VS.NET code) and NEW (GNU gcc) and encourage projects to migrate over.
The end game is a turnkey install that any non-tech user can safely and reliably perform on any platform after having only needed to install NodeJS.
@rlidwka A Vagrant VM is serious overkill for running what is often a small JS script that would then be disconnected from the local Windows filesystem and the rest of the Window OS.
tohagan commentedon Aug 12, 2014
Was thinking the other day that perhaps a cloud based compile of binary node packages might be the way to go as it would ensure a uniform CI based build environment and remove the need for end users to install compilers or the joyent team to include them in NodeJS CORE. The Node framework could provide this as part of the npmjs.org service (it compiles package C++ code, runs unit tests and signs binaries that can then be published to npm registry and npm client checks digital signatures of binaries.
A risk factor is that you can't confirm that source matches binary so I'm recommending this type of service uses digital signatures that include a reference to the GIT repo + version that code was built from so we can at least correlate the two. Another significant risk factor is running the untrusted unit tests in the build service.
I also found this app that looks like it's implementing some of this ... but I wonder if it really belongs as part of the npm registry?
If this app is solid and security-wise safe enough, it might be viable to solve the ws problem in the short term.
48 remaining items
jloramas commentedon Oct 18, 2015
+1 for removing VS dependency
LeeU1911 commentedon Nov 5, 2015
I found this topic by digging through a whole lot of pages in order to solve my problem with installing mongodb via npm. It also requires VCBuild.exe in the build path, aka Visual Studio dependency.
As a Java developer, my organization obviously won't understand why I need to get a ~4GB+ copy of Visual Studio if I don't have a valid reason. This problem got worse since now I'm working on a new computer in a remote location and I have to go through all the pains with this again.
So I also vote +1 for removing VS dependency.
Sorry for the long rant.
justinshea commentedon Nov 8, 2015
+1
cheoAlejo commentedon Nov 17, 2015
+1
AngelEyesChina commentedon Nov 17, 2015
+1
akreienbring commentedon Dec 7, 2015
+1
Marcoevich commentedon Dec 14, 2015
+1 for removing VS dependancy
bexoss commentedon Dec 15, 2015
+1 It should be.
jacolaisen commentedon Dec 28, 2015
+1 you shall remove VS dependancy and leave the door open!
er-sandeepsingh commentedon Dec 28, 2015
+1
cherryliu commentedon Jan 5, 2016
+1. Thanks @tohagan! I thought I was blocked on build error on windows too, although sockets all works well. +1 to the point that the build errors could be changed to warnings instead! Socket.io is really an amazing tool that opens so many doors, it just needs this touchup on installation UX.
tohagan commentedon Jul 2, 2017
Was delighted this week to discover that the latest npm performs binary installs and creates a package-lock.json containing SHA checksums. Excellent work!
After reviewing the breaking changes in npm v5.0, I recommend you
npm -g install npmtoday!Mithgol commentedon Jul 12, 2017
Does the latest npm really perform binary installs? What part of http://blog.npmjs.org/post/161081169345/v500 does mention it?
(It seems to me that the only place where http://blog.npmjs.org/post/161081169345/v500 mentions any binary is the phrase “One binary to rule them all:
./cli.jshas been removed in favor of./bin/npm-cli.js”.That's why I ask.)
gibfahn commentedon Jul 13, 2017
That refers to the npm "binary" that you run, there's less "script calling script calling script" in
npm5. It's not actually a binary (compiled) file though.As for native modules, npm will compile them using the system C++ compiler. However a lot of modules include prebuilt binaries for most platforms, using something like node-pre-gyp. Other than having a
package-lock.jsonthat notes down the dependency versions, I don't think npm5 is doing anything different.So if you want some package that's written in C++ to magically install without compiling, then you need the module developers to include prebuilt binaries in their npm package.
There have been a number of improvements for people using Node on Windows (many led by Microsoft).
Also this repo is archived, issues with Node should be raised in https://github.com/nodejs/node. The only part of this that might be worth raising in
nodejs/nodeis the discussion about either bundling a compiler, or moving tollvm( @bnoordhuis how did that go?)For general questions about node, please raise issues in https://github.com/nodejs/help instead.
I'll close this.