time parsing in dmesg log

This is the place for queries that don't fit in any of the other categories.

Re: time parsing in dmesg log

Postby Yoriz » Mon Mar 04, 2013 6:43 am

raohiral wrote:Thanks for help..

I tried giving string and number but still getting this error,

cur_state['kernel_time_stamp'] = ' '

error:

Traceback (most recent call last):
File "C:\Bug2Go-scripts\pm_log_parser_new.py", line 1328, in <module>


float on an empty string will give an error
Code: Select all
float('')
ValueError: could not convert string to float:

if thats what you mean, if not that then you didn't actually give an error at all.
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 724
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: time parsing in dmesg log

Postby raohiral » Mon Mar 04, 2013 5:50 pm

yeah thats what the issue is..

I am getting this error,

float('')
ValueError: could not convert string to float:


But I will have to return empty string because in below function it will parse time and body from the log file,

Code: Select all
def roll(fobj_in, fobj_out):
    cur_state = __init_cur()
    live_sessions = dict()
    live_sessions['full'] = None
    live_sessions['discharge'] = None
    live_sessions['charge'] = None
    live_sessions['active'] = None
    live_sessions['suspend'] = None
    live_sessions['wakeup'] = None # started from wakeup_wakelock, ended
    live_sessions['last_active'] = None

    try:
        l = fobj_in.readline()
     
        while (len(l)):
            t,b = time_and_body(l)
            if t is None:
                cur_state['kernel_time_stamp'] = t
               
                if not live_sessions['full']:
                    live_sessions['full'] = FullLogSession(
                        start = t,
                       
                        start_time = __current_time(cur_state))
                   
                    live_sessions['discharge'] = DischargeSession(
                        start = t,
                        start_time = __current_time(cur_state))
                for k in REGEX.keys():
                    r,f = REGEX[k]
                    m = r.match(b)
                    if m is not None:
                        if f in globals().keys():
                            hook = globals()[f]
                            hook(live_sessions, cur_state, m)
            l = fobj_in.readline()
        # close all live sessions
        __close_sessions(live_sessions, cur_state)
        live_sessions['full'].end = cur_state['kernel_time_stamp']
        live_sessions['full'].end_time = __current_time(cur_state)
        live_sessions['full'].duration = __duration(live_sessions['full'])\
            + live_sessions['full'].rtc_only
    except:
        __debug_print_all(live_sessions, cur_state)
        raise
    if not TECH and not VERBOSE:
        if cur_state['cpu1_on'] > 0:
            live_sessions['full'].cpu1_time += __current_time(cur_state) - cur_state['cpu1_on']
        if cur_state['charging_start'] > 0:
            live_sessions['full'].usb_time += __current_time(cur_state) - cur_state['charging_start']
    return live_sessions['full']



if you see its taking time stamp from here,

if t is None:
cur_state['kernel_time_stamp'] = t

I am not sure..where i am doing wrong..
raohiral
 
Posts: 18
Joined: Fri Mar 01, 2013 3:59 pm

Re: time parsing in dmesg log

Postby raohiral » Mon Mar 04, 2013 5:58 pm

ok I ran with

Code: Select all
KERNEL_TIME_STAMP = re.compile(r'(\d{2}):(\d{2}):(\d{2}) (\d{3}) (.*)')
   
LOGCAT_TIME_STAMP =\
    re.compile('^(\d{2})-(\d{2}) (\d{2}):(\d{2}):(\d{2})\.(\d{3}) (.*)')
def time_and_body(line):
     
    n = KERNEL_TIME_STAMP.search(line)
    if n:
        groups = n.groups()
        return '{}.{}.{}.{}'.format(*groups[:4]), groups[4]
    return (None, None)
   


and gave print

Code: Select all
try:
        l = fobj_in.readline()
     
        while (len(l)):
            t,b = time_and_body(l)
            print time_and_body(l)


Its throwing (None,None)..
raohiral
 
Posts: 18
Joined: Fri Mar 01, 2013 3:59 pm

Re: time parsing in dmesg log

Postby Yoriz » Mon Mar 04, 2013 6:44 pm

(None,None) is what you have your function time_and_body return when the reg-ex does not match, What do you want it to do when the line it reads is not a matching time stamp?
I also noticed in your previous post the
Code: Select all
if t is None:

was
Code: Select all
if t is not None:

in an earlier post, have you altered it by mistake ?.
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 724
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: time parsing in dmesg log

Postby raohiral » Mon Mar 04, 2013 7:06 pm

(None,None) is what you have your function time_and_body return when the reg-ex does not match, What do you want it to do when the line it reads is not a matching time stamp?

Yeah you are right, when it does not match it returns None, but I want to match re-exp and return below result,


line = ("02-28 16:25:50.184 2941 2941 I KERNEL : [ 5816.432472,1] mtp_open")

I need below parsing result..
('16.25.50.184', ' 2941 2941 I KERNEL : [ 5816.432472,1] mtp_open')

so I used this code,

Code: Select all
KERNEL_TIME_STAMP =\
    re.compile('^(\d{2})-(\d{2}) (\d{2}):(\d{2}):(\d{2})\.(\d{3}) (.*)')

def time_and_body(line):
   
    n = KERNEL_TIME_STAMP.match(line)
       
    if n is None:
        return (None,None)
    k_time = n.groups()[2]+'.'+n.groups()[3]+'.'+n.groups()[4]+'.'+n.groups()[5]
    #print k_time
    return (k_time,n.groups()[6])


And It was returning,
('16.25.50.184', ' 2941 2941 I KERNEL : [ 5816.432472,1] mtp_open')...
raohiral
 
Posts: 18
Joined: Fri Mar 01, 2013 3:59 pm

Re: time parsing in dmesg log

Postby raohiral » Mon Mar 04, 2013 7:43 pm

I am passing cur_state['kernel_time_stamp']=18.00.16.832

to function,

Code: Select all
def __current_time(c):
    return float(c['kernel_time_stamp'])


giving me error,

Traceback (most recent call last):
File "C:\Bug2Go-scripts\pm_log_parser_new.py", line 1341, in <module>
f = roll(fobj_in, fobj_out)
File "C:\Bug2Go-scripts\pm_log_parser_new.py", line 950, in roll
start_time = __current_time(cur_state))
File "C:\Bug2Go-scripts\pm_log_parser_new.py", line 37, in __current_time
return float(c['kernel_time_stamp'])
ValueError: invalid literal for float(): 18.00.16.832
raohiral
 
Posts: 18
Joined: Fri Mar 01, 2013 3:59 pm

Re: time parsing in dmesg log

Postby Yoriz » Mon Mar 04, 2013 7:51 pm

You cant turn 18.00.16.832 into a float there is way to many '.'
Code: Select all
float('18.00.16.832')
ValueError: invalid literal for float(): 18.00.16.832
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 724
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: time parsing in dmesg log

Postby raohiral » Mon Mar 04, 2013 10:01 pm

so what should I use if it wont be float?

Coz I have to convert into number in order to get time..

Thanks
raohiral
 
Posts: 18
Joined: Fri Mar 01, 2013 3:59 pm

Re: time parsing in dmesg log

Postby Yoriz » Mon Mar 04, 2013 10:46 pm

I have no idea what your overall code is doing, i only see bits and pieces and errors that you are getting, only you know what is meant to be going on. I'm trying the best i can to help but i have no idea why you would want to try and turn 18.00.16.832 into a float or what other format you would want it to be in, you need to try and figure out what it is your trying to do.
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 724
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: time parsing in dmesg log

Postby raohiral » Mon Mar 04, 2013 10:51 pm

Hey sorry to bug you..

But thats what I did..I didnt convert it into anything...I tried it and it removed my error..

Thanks for helping me out through out this..

I really appreciate your help.

Thanks
raohiral
 
Posts: 18
Joined: Fri Mar 01, 2013 3:59 pm

Re: time parsing in dmesg log

Postby raohiral » Tue Mar 05, 2013 6:08 am

hi can you suggest..what would be regular expression to parse "49799.938157" from below line..


line = '02-16 09:41:03.614 17880 17880 I KERNEL : [49799.938157,0] PM: Entering mem sleep'

i have used this reg-exp, seems like its not reading whole file,

Code: Select all
new = re.compile('^(.*)\[ *(\d+)\.(\d{6})[^\]]*\] (.*)')

is there something wrong?
raohiral
 
Posts: 18
Joined: Fri Mar 01, 2013 3:59 pm

Re: time parsing in dmesg log

Postby Yoriz » Tue Mar 05, 2013 7:43 am

Yours gave
Code: Select all
('02-16 09:41:03.614 17880 17880 I KERNEL : ', '49799', '938157', 'PM: Entering mem sleep')

Change the brackets to include the '.'
Code: Select all
new = re.compile(r'^(.*)\[ *(\d+\.\d{6})[^\]]*\] (.*)')

Will then give
Code: Select all
('02-16 09:41:03.614 17880 17880 I KERNEL : ', '49799.938157', 'PM: Entering mem sleep')

if thats what you want as you never said if you just wanted '49799.938157', you should learn to discribe what your getting and what you want better then the help you get is more likely to be what you want :D
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 724
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: time parsing in dmesg log

Postby raohiral » Tue Mar 05, 2013 4:49 pm

Thanks so much,


actually I needed result like this,

Code: Select all
('02-16 09:41:03.614 17880 17880 I KERNEL : ', '49799', '938157', 'PM: Entering mem sleep')


I got using reg-exp,

Code: Select all
re.compile(r'^(.*)\[ *(\d+)\.(\d{6})[^\]]*\] (.*)')


and wanted to merge using,

Code: Select all
n.groups()[1]+'.'+n.groups()[2],n.groups()[3]


I got result what I wanted to see but I had issue where it was not reading the whole file somewhere it broke the line and threw error. I wanted to make sure if my reg-exp is wrong, but I feel thats not the case..Its broken @ some other function. So, trying to debug now what went wrong in code.

but thanks I will make sure to write expected result and result I got from execution.
raohiral
 
Posts: 18
Joined: Fri Mar 01, 2013 3:59 pm

Previous

Return to General Coding Help

Who is online

Users browsing this forum: No registered users and 4 guests