[Python] Customer Base?

[Leitz]

To the larger point of Python versus whatever, a nice reusable module with pydoc strings and some sort of default behavior if called directly let’s end users do what they please. The front end gets to be decided by an end user. That being said the Tk GUI toolkit is often distributed alongside Python installs. OSX ships with Tk 8.5.9 IIRC and many Linux distros also ship TCL/Tk out of the box as well. On Windows you’d likely distribute via py2exe so it’s immaterial.

While a lot of folks on the e-mail lists decry Tk (Ruby, Python, etc), it is still a decently standard interface. TCL isn't too bad either.

Using a language like Go would likely end up with more problems as the pool of contributors is smaller and the tooling and packaging for non-Unix systems is not very mature or robust. The cross section of people with adequate Go experience, experience with writing cross-platform GUI apps in Go, and people wanting and willing to work on Traveller apps is likely to be minuscule.

Like they say, "There's always one." :rofl: Not sure how the small pool is an issue. The reason I had brought it up was the cross-platform usage. The idea seems to be there but I haven't tested it. "Mature" is a mixed blessing; it can also mean old and antiquated. Robust, well, needs testing.

That said, if the Python to App option works then it works. It might help if we look at what needs packaging as an app and what will remain a non-app library. The various world generation systems seem like apps, the base_tools.py is just a library.

Another option is a web based platform and I seem some folks using Flask or similar. That eliminates the portability issue for module installations but only works when web-connected.

 

[Leitz]

And yes, I have been responsible for running REST servers, SOAP servers, and other types of servers. I can tell you from personal experience that because REST servers are easier to test they are much more solid that other types of servers. Same with ease of troubleshooting; makes for less downtime.

I'll take the "follow" in this discussion. What can I do to help those of you who want to do stuff with web services? Do we have decisions to make for "standards"; Flask vs Bottle vs Django, or whatever. Use a Digital Ocean droplet or someone's web host?

Let me know how to help.

 

[ShawnDriscoll]

To the larger point of Python versus whatever, a nice reusable module with pydoc strings and some sort of default behavior if called directly let’s end users do what they please. The front end gets to be decided by an end user. That being said the Tk GUI toolkit is often distributed alongside Python installs. OSX ships with Tk 8.5.9 IIRC and many Linux distros also ship TCL/Tk out of the box as well. On Windows you’d likely distribute via py2exe so it’s immaterial.

Go use is climbing. But not at huge margins. It's a "need to use" only language for now. It's in the under 1% usage crowd. Python is in the 7% usage crowd.

So PyGTK comes with Python now? Maybe I'm confused by the Tk GUI term. Tkinter had the head-start for the longest time with GUI users because it came with Python already. Or course, Qt is still king of the GUI tools. But it is showing a leveling-off in the last 6 months. Maybe Kivy is grabbing some of that pie? Or a web GUI is slowly replacing local GUI usage for Python apps, especially on hand-helds?

if the Python to App option works then it works. It might help if we look at what needs packaging as an app and what will remain a non-app library. The various world generation systems seem like apps, the base_tools.py is just a library.

Or just have a Setup.py file on the Git for people that want to turn their downloaded code into an EXE if they want.

 

[giant.robot]

So PyGTK comes with Python now? Maybe I'm confused by the Tk GUI term. Tkinter had the head-start for the longest time with GUI users because it came with Python already. Or course, Qt is still king of the GUI tools. But it is showing a leveling-off in the last 6 months. Maybe Kivy is grabbing some of that pie? Or a web GUI is slowly replacing local GUI usage for Python apps, especially on hand-helds?.

Tk as in Tkinter is what I meant. The actual name of the GUI toolkit is Tk, Tkinter is just the Python binding library for the Tk C library. Sorry for the confusion.

If a server-side solution is interesting Flask is probably the easiest thing to get up and running (I work with Flask all day at work). It’s specifically tuned for web apps with very little configuration needed. It’s also very “low magic” making it simple to understand. The Jinja templating language is also very easy to understand and work with. It’s all very straight forward.

 

[joshualevy]

Do we have decisions to make for "standards"; Flask vs Bottle vs Django, or whatever. Use a Digital Ocean droplet or someone's web host?

I don't think we need to make standards, but I also think that there are some standards that would be helpful. My goal in these standards are NOT that everyone do the same thing, but rather that different people not be slightly different for no good reason. If two people want to do different things, then they will, and that's fine. But if they want to do the same thing, then standards should help them interact.

I think that there are two ways to write programs in the modern world. You can have programs that run on one computer. They are "classic" programs. You have a library that does something, you have a GUI that the user sees, and they work together. The big questions here are what OS, what language and what GUI system you use. The second system is you have a server and a client which are often on different machines. These are generally web sites of some kind. The big question for these systems are what is the interface to your server, and how do you present this information to the user.

So with that as prolog, here are some of my thoughts:

* I think that standards are most important for communication, data interchange, and (maybe) user interface. Conversely, I think that things like server software and web host don't really need to be standardized. As an example, lets say that I create a web server based on Bottle, and you create one based on Flask. Who cares? People can easily use your service or mine, or both. I can use your's. It just doesn't matter.

* For classic programs, the important standardization issues are OS, language, and GUI, and to be honest I don't want to discuss those. I don't think it a good idea (or possible) to convince someone to use a language, an OS or a GUI that they don't want to use. So everyone should use what they want, and that limits their sharing, but there is no way around it. The Python folks can share libraries, and so on.

* But for server/client programs the important standardization is the communications layer between the server and the client. And if you standardize (at least a little) then it means that every person who writes a client can take advantage of every server: and that is very beneficial!

* Also, it means that I can write my server in any language I want and run it on any OS, and everyone can take advantage of it. Indeed, one client can get information from many different servers.

* So I would try to come up with some guidelines or standardization for server interfaces.

I'm running out of time to write this, so I'm going to be very brief about the standardizations I'd like to see:

1. Start out with a list of all servers of Traveller data on the web. That alone would help people avoid duplication and quickly create new clients based on existing servers. This would be strictly descriptive. No rules. Just one place to go to see what is out there.

2. I think that REST is the best server interface system, so I'd have a guideline that servers of Traveller data should have REST interfaces.

3. A REST interface must return data in some format. I can see two ways to move forward here: (a) I think the best data format is json, so I would recommend that servers of Traveller data should return data in json format. (b) another option to say that all servers should take one URL argument which specifies what data format should be returned. So people could add "format=xml" or "format=ini" to their URL and get their data in whatever format they want. Obviously, these can be combined where json is the default but other formats can be specified.

4. It would be useful to have a standard place to go for a human readable description of the server. So maybe we have a guideline that for each server of Traveller data, there should be a web page "/help" off that server which describes the services provided by that server.

5. It might be nice to have some standard arguments for servers. Maybe something like "count=N" to tell the server how many items to return. Obviously, that would not make sense in some cases, but the idea is that servers would not use "number" or "repeat" or "num" or anything else. If you supported many return values, then you would use the "count" argument and nothing else.

You can see some of these ideas here: http://joshualevy.pythonanywhere.com
take a look at these URLs:
http://joshualevy.pythonanywhere.com/help
http://joshualevy.pythonanywhere.com/yt?format=json

The "human readable" version of this server is here:
http://joshualevy.pythonanywhere.com/yt?format=htmlPage
http://joshualevy.pythonanywhere.com/yt?format=html

Joshua Levy

 

[Leitz]

"It was a dark and stormy night..."

Actually, it was a rather average night; my wife was out doing photography, the dogs were keeping the floor warm, and I stared at two CSV files with over-lapping data that I wanted to put into MongoDB.

A couple hours, a few dozen lines of python, and a break to re-read the material from my MongoDB book later and 'ta-da'!

MongoDB the company has free classes. M101P uses Python and Bottle with MongoDB and I'm likely to sign up for the next one now that I have a little confidence in my ability to grok bottle and python more. I've never really been good at CRUD operations. Post class we can talk about Flask or whatever. In using MongoDB I'm +1 for json output.

Sadly, now it's time for work...

I put the code in '/examples'. Part of the issue is that it's specialized for a specific data layout and requires pymongo and a MongoDB backend. Probably needs tests and exception handling as well. :rofl:

 

[ShawnDriscoll]

I have sqlite3, but I'm not a fan of SQL. JSON anything sounds interesting. Looking at PyMongo now.

 

[Leitz]

JSON is finky about commas and such but is very portable. MongoDB uses JavaScript for it's internals and SQL interface and natively does JSON. I think you'll enjoy it.

 

[ShawnDriscoll]

Ha! Got sent down a rabbit hole with this one.

Now I have:
pymongo-2.9.5-py2.5-win32.egg
sphinx-0.4.3-py2.5.egg
jinja-1.2-py2.5-win32.egg
pygments-1.4-py2.5.egg

and docutils-0.10-py2.5.egg

installed.

 

[whartung]

sqlite is more an embedded store, and much easier for casual use that spooling up a MongoDB server.

 

[egor045]

I have sqlite3, but I'm not a fan of SQL. JSON anything sounds interesting. Looking at PyMongo now.

PyMongo's API is kind of horrible (I poked around at it a while ago for a side project). Once you get desensitized to that it's ... OK.

It'd be great if there was a NoSQL equivalent of SQLite that you could cheaply/easily stuff into an application package. Having to deploy MongoDB adds a bit to the overhead.

 

[ShawnDriscoll]

sqlite is more an embedded store, and much easier for casual use that spooling up a MongoDB server.

So far, I'm liking PyMongo's db commands a whole lot more than I ever did sqlite3's db commands (not an SQL fan at all). Playing with the sample dragons database right now from Leitz. This might actually get me to write a database app. Would always lose interest before with other implementations for Python.

PyMongo's API is kind of horrible (I poked around at it a while ago for a side project). Once you get desensitized to that it's ... OK.

It'd be great if there was a NoSQL equivalent of SQLite that you could cheaply/easily stuff into an application package. Having to deploy MongoDB adds a bit to the overhead.

MongoDB would just be for my own personal Traveller use. I don't deploy database client/server apps. Now to fit my CSV file into a Mongo database. My Git (just google my name and github) shows what kind of data my CSV has in it.

 

[Leitz]

It'd be great if there was a NoSQL equivalent of SQLite that you could cheaply/easily stuff into an application package. Having to deploy MongoDB adds a bit to the overhead.

mongoexport can return JSON. Thus you can have your MongoDB server at home, manipulate the data, and then push a JSON file up to the web site for read-only type use. That's my plan, anyway.

 

[egor045]

mongoexport can return JSON. Thus you can have your MongoDB server at home, manipulate the data, and then push a JSON file up to the web site for read-only type use. That's my plan, anyway.

A little bit of free association (Perl associative array bound to a file -> key-value store -> Google search) brought me to PickleDB (https://pythonhosted.org/pickleDB/). Not quite a full-featured embeddable NoSQL database, but useful all the same. Thanks for sparking that line of thinking

 

[Leitz]

egor045, happy to help!

The main reason I don't use things like pickle and shelve is that they are python only, far as I know. With json, sqlite, and Mongo we can share data with other languages.

 

[egor045]

egor045, happy to help!

The main reason I don't use things like pickle and shelve is that they are python only, far as I know. With json, sqlite, and Mongo we can share data with other languages.

PickleDB has an open issue to rename it to jsonDB or similar. Looking at the code, the file format is JSON.

 

[Leitz]

Another question for the group: Merges?

Started the morning hoping to pull from MongoDB and import the data into my Character class. Found the UTF-8 issue wasn't really solved and worked on that. In the process I've changed from print(char) to char.display(). The "__str__" method didn't handle UTF-8 in Python 2.

The question is simple, what sort of process do we use to verify code before a merge? My code passes tests but my test suite isn't all inclusive. And I got up way to early. I'd like to get at least one person to check the new code before merging it into the master branch. Any takers?

The corollary question is how long to wait for review? If others start to pull and push then unmerged code can become stale. We *know* the py_tools repo will attract international attention and hundreds of developers, right? :coffeesip:

Just my thoughts before I zonk out. No rush on code review, more the idea of it.

 

[egor045]

Another question for the group: Merges?

Started the morning hoping to pull from MongoDB and import the data into my Character class. Found the UTF-8 issue wasn't really solved and worked on that. In the process I've changed from print(char) to char.display(). The "__str__" method didn't handle UTF-8 in Python 2.

The question is simple, what sort of process do we use to verify code before a merge? My code passes tests but my test suite isn't all inclusive. And I got up way to early. I'd like to get at least one person to check the new code before merging it into the master branch. Any takers?

The corollary question is how long to wait for review? If others start to pull and push then unmerged code can become stale. We *know* the py_tools repo will attract international attention and hundreds of developers, right? :coffeesip:

Just my thoughts before I zonk out. No rush on code review, more the idea of it.

+1 for code reviews. They're good for code quality and for both reviewers and reviewees.

I'm happy to do a review for your new code.

In order to figure out how long to wait, that depends on how many reviewers we have, how often reviews are requested, and the work involved in doing the review. Anybody got any estimates?

 

[Leitz]

Well, now that I think I learned to spell, the more_skill_options and dragon_to_char branches should be good for a review. If they check out I'll merge and fix issues.

Thanks!

 

[Leitz]

egor045 ran into an issue based on Python 2.6 testing being different than 2.7 or 3.x. Something we need to consider; I know there are valid reasons for staying on 2.6 and earlier. This may be a simple fix but as a team I'd recommend we think about "What level of effort is spent on compatibility?"

My code works on Python 2.6 and 3.6. Of course, my code is really basic stuff and even with that I have to work to support py2 and py3.

Thoughts?