New Python user: suitable as a database &modelling platform?

This is the place for queries that don't fit in any of the other categories.

New Python user: suitable as a database &modelling platform?

Postby AEA » Thu Apr 18, 2013 6:05 pm

Hello Python-forum.org

As with most things it is necessity that has brought/forced me to consider switching my modelling from excel (I know I know, it just kind of evolved this way) to python.

My previous project using excel, was modelling the likelihood of getting different types of cancer, there were a huge number of parameters and I had a very large number of patients record/history. I believe I pushed excel close to its limit with an excel file in excess of 500 mb and would run solver based evolutionary algorithms for weeks at a time in order to increase the accuracy of the prediction. However as I am sure you are all aware (or perhaps not?) using excel is a mile from ideal, the sheet became unstable as a result of the size I believe and because of the time required to perform the iterations using the evolutionary algorithm the addition of further parameters to enhance the predictive capabilities was averted.

So my questions to the Python coders community are these:

Is Python the correct platform to undertake this project?
Can python be used as a database and then particular queries called in order to analyse them?
Is it realistic for someone who is just starting in python to undertake a project like this?

Please feel free to add any notes to your thoughts on this topic, or words of guidance, or possibly the way in which you feel i should go about said problem.

Many thanks,

AEA
AEA
 
Posts: 32
Joined: Thu Apr 18, 2013 11:37 am

Re: New Python user: suitable as a database &modelling platf

Postby setrofim » Thu Apr 18, 2013 6:38 pm

AEA wrote:Is Python the correct platform to undertake this project?

Sure. It's not the correct platform, but it's a correct platform.
AEA wrote:Can python be used as a database and then particular queries called in order to analyse them?

Python is a programming language; a database is something different. But yes, you can use Python to interact with databases, and to analyse the results of querying a database.
AEA wrote:Is it realistic for someone who is just starting in python to undertake a project like this?

It's ambitious, but not unreasonable, if you're prepared to invest the time. Start here and go from there. Post on this forum if you get stuck.
setrofim
 
Posts: 288
Joined: Mon Mar 04, 2013 7:52 pm

Re: New Python user: suitable as a database &modelling platf

Postby AEA » Thu Apr 18, 2013 6:55 pm

Hello Setrofim,

Many thanks for the reply.

It's not the correct platform, but it's a correct platform.

Perhaps I should rephrase this, If you were me would Python be your choice of platform for such a project?

It's ambitious, but not unreasonable

That's what I like to hear. Its all one step at a time kind of thing. I am willing to put the time in.

But yes, you can use Python to interact with databases, and to analyse the results of querying a database.

Is there a recommended database which can perform queries more effectively and efficiently? Again to rephrase, If you were me which type of database would you select for said project?

Start here and go from there. Post on this forum if you get stuck.

Thanks for providing me with a starting point.

Many thanks

AEA
AEA
 
Posts: 32
Joined: Thu Apr 18, 2013 11:37 am

Re: New Python user: suitable as a database &modelling platf

Postby setrofim » Thu Apr 18, 2013 8:13 pm

AEA wrote:Perhaps I should rephrase this, If you were me would Python be your choice of platform for such a project?

Yes. (but then, I answer questions on a Python forum, so what do you expect? ;)) Python has long been the language of choice for scientific and analytical computing and has a lot of support in this area (there is even an annual confrence that focuses on applications of Python in data modeling and analysis).

AEA wrote: Its all one step at a time kind of thing.

Yup, absolutely. There is a lot that Python can offer you in this area; way too much to take in at once (there are libraries, tools, frameworks, environments, etc). But you can achieve a lot even with a basic understanding of the laguage, and then continuosly refine your process as you learn new stuff.

AEA wrote:Is there a recommended database which can perform queries more effectively and efficiently? Again to rephrase, If you were me which type of database would you select for said project?

Databases are a rather complicated subject in their own right. Which to use would depend partly on your requirements and partly on subjective opinion. For now, I suggest you look at sqlite to begin with. Python's standard library has a module for working with it. However, depending on what you are doing, you may not even want a database, and use a flat file (e.g. in JSON format) instead.
setrofim
 
Posts: 288
Joined: Mon Mar 04, 2013 7:52 pm

Re: New Python user: suitable as a database &modelling platf

Postby AEA » Fri Apr 19, 2013 12:08 am

Many thanks setrofim, its nice to know this forum has some friendly members who aren't dismissive of bury eyed newby :)

Hopefully the internet will provide a lot of the answers I require without pestering you lot on here until I'm in the deep end (no promises though)

Many thanks for your help.

Kind regards

AEA
AEA
 
Posts: 32
Joined: Thu Apr 18, 2013 11:37 am


Return to General Coding Help

Who is online

Users browsing this forum: Bing [Bot], snippsat and 6 guests