Rrerku

Are there any examples of a variable being normally distributed that is *not* due to the Central Limit Theorem?

Alternative to sending password over mail?

Why is consensus so controversial in Britain?

How did the Super Star Destroyer Executor get destroyed exactly?

How dangerous is XSS?

Do scales need to be in alphabetical order?

Do UK voters know if their MP will be the Speaker of the House?

Why didn't Miles's spider sense work before?

How do conventional missiles fly?

How do I handle a potential work/personal life conflict as the manager of one of my friends?

Is there an expression that means doing something right before you will need it rather than doing it in case you might need it?

What does the expression "A Mann!" means

Size of subfigure fitting its content (tikzpicture)

Can my sorcerer use a spellbook only to collect spells and scribe scrolls, not cast?

Should I tell management that I intend to leave due to bad software development practices?

Avoiding direct proof while writing proof by induction

Gatling : Performance testing tool

Arrow those variables!

How can I deal with my CEO asking me to hire someone with a higher salary than me, a co-founder?

One verb to replace 'be a member of' a club

What is a romance in Latin?

Why doesn't using multiple commands with a || or && conditional work?

Why are the 737's rear doors unusable in a water landing?

How badly should I try to prevent a user from XSSing themselves?

Watching something be piped to a file live with tail

How to continue running a program inspite of killing the shell which invoked itLinux terminal, my program only resumes after scrolling the printoutIs there a command in Linux which waits till it will be terminated?Ctrl+c in a sub process is killing a nohup'ed process earlier in the scriptCtrl-C'd an in-place recursive gzip - is this likely to have broken anything?Getting output of another script while preserving line-breakssocat and rich terminal againScript executed as other user from root creates files in wrong directory (root)Dealing with Ctrl+Z in UnixHow to pass Ctrl+C to script called from batch job

5

I have a python program which is, slowly, generating some output.

I want to capture that in a file, but I also thought I could watch it live with tail.

So in one terminal I'm doing :

python myprog.py > output.txt

and in another terminal :

tail -f output.txt

But it seems like the tail isn't showing me anything while the python program is running. If I hit ctrl-c to kill the python script, suddenly the tail of output.txt starts filling up. But not while the python is running.

What am I doing wrong?

linux command-line pipe

share|improve this question

asked 5 hours ago

interstarinterstar

335311

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  
  How about python myprog.py | tee output.txt instead?
  
  – n8te
  5 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @n8te tee might show the same problem if the program isn't flushing the output buffer regularly. This needs flush() and tee.
  
  – JPhi1618
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @JPhi1618 - you could be right but I just tested it with 800MB of data output and it didn't skip a beat.
  
  – n8te
  2 hours ago
  

  
  
  
  

add a comment | 

5

I have a python program which is, slowly, generating some output.

I want to capture that in a file, but I also thought I could watch it live with tail.

So in one terminal I'm doing :

python myprog.py > output.txt

and in another terminal :

tail -f output.txt

But it seems like the tail isn't showing me anything while the python program is running. If I hit ctrl-c to kill the python script, suddenly the tail of output.txt starts filling up. But not while the python is running.

What am I doing wrong?

linux command-line pipe

share|improve this question

asked 5 hours ago

interstarinterstar

335311

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  
  How about python myprog.py | tee output.txt instead?
  
  – n8te
  5 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @n8te tee might show the same problem if the program isn't flushing the output buffer regularly. This needs flush() and tee.
  
  – JPhi1618
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @JPhi1618 - you could be right but I just tested it with 800MB of data output and it didn't skip a beat.
  
  – n8te
  2 hours ago
  

  
  
  
  

add a comment | 

I have a python program which is, slowly, generating some output.

I want to capture that in a file, but I also thought I could watch it live with tail.

So in one terminal I'm doing :

python myprog.py > output.txt

and in another terminal :

tail -f output.txt

But it seems like the tail isn't showing me anything while the python program is running. If I hit ctrl-c to kill the python script, suddenly the tail of output.txt starts filling up. But not while the python is running.

What am I doing wrong?

linux command-line pipe

share|improve this question

asked 5 hours ago

interstarinterstar

335311

python myprog.py > output.txt

tail -f output.txt

share|improve this question

asked 5 hours ago

interstarinterstar

335311

share|improve this question

asked 5 hours ago

interstarinterstar

335311

asked 5 hours ago

interstarinterstar

335311

asked 5 hours ago

interstarinterstar

335311

2 Answers
2

active

oldest

votes

8

You may also need to explicitly flush the buffer for it to get piped upon generation. This is because output is typically only printed when the pipe's buffer fills up (which is in kilobytes I belive), and when the stdin message ends. This is probably to save on read/writes. You could do this after every print, or if you are looping, after the last print within the loop.

import sys
...
print('Some message')
sys.stdout.flush()

share|improve this answer

edited 51 mins ago

user2313067

2,1001911

answered 5 hours ago

DaveyDavey

961

New contributor

Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  If you have read this far, please don't be thinking of closing and re-opening the file to do this, the seeks will be a problem, especially for very large files. (I've seen this done!).
  
  – mckenzm
  1 hour ago
  

  
  
  
  

add a comment | 

5

Instead of trying to tail a live file, use tee instead. It was made to do exactly what you're trying to do.

From man tee:

tee(1) - Linux man page

Name tee - read from standard input and write to standard output and files

Synopsis

tee [OPTION]... [FILE]...

Description

Copy standard input to each FILE, and also to standard output.

-a, --append 
 append to the given FILEs, do not overwrite 
-i, --ignore-interrupts 
 ignore interrupt signals 
--help 
 display this help and exit 
--version
 output version information and exit

If a FILE is -, copy again to standard output.

So in your case you'd run:

python myprog.py | tee output.txt

share|improve this answer

answered 5 hours ago

n8ten8te

5,19272234

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "3"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1421123%2fwatching-something-be-piped-to-a-file-live-with-tail%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

8

You may also need to explicitly flush the buffer for it to get piped upon generation. This is because output is typically only printed when the pipe's buffer fills up (which is in kilobytes I belive), and when the stdin message ends. This is probably to save on read/writes. You could do this after every print, or if you are looping, after the last print within the loop.

import sys
...
print('Some message')
sys.stdout.flush()

share|improve this answer

edited 51 mins ago

user2313067

2,1001911

answered 5 hours ago

DaveyDavey

961

New contributor

Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  If you have read this far, please don't be thinking of closing and re-opening the file to do this, the seeks will be a problem, especially for very large files. (I've seen this done!).
  
  – mckenzm
  1 hour ago
  

  
  
  
  

add a comment | 

8

You may also need to explicitly flush the buffer for it to get piped upon generation. This is because output is typically only printed when the pipe's buffer fills up (which is in kilobytes I belive), and when the stdin message ends. This is probably to save on read/writes. You could do this after every print, or if you are looping, after the last print within the loop.

import sys
...
print('Some message')
sys.stdout.flush()

share|improve this answer

edited 51 mins ago

user2313067

2,1001911

answered 5 hours ago

DaveyDavey

961

New contributor

Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  If you have read this far, please don't be thinking of closing and re-opening the file to do this, the seeks will be a problem, especially for very large files. (I've seen this done!).
  
  – mckenzm
  1 hour ago
  

  
  
  
  

add a comment | 

You may also need to explicitly flush the buffer for it to get piped upon generation. This is because output is typically only printed when the pipe's buffer fills up (which is in kilobytes I belive), and when the stdin message ends. This is probably to save on read/writes. You could do this after every print, or if you are looping, after the last print within the loop.

import sys
...
print('Some message')
sys.stdout.flush()

share|improve this answer

edited 51 mins ago

user2313067

2,1001911

answered 5 hours ago

DaveyDavey

961

New contributor

Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

import sys
...
print('Some message')
sys.stdout.flush()

share|improve this answer

edited 51 mins ago

user2313067

2,1001911

answered 5 hours ago

DaveyDavey

961

New contributor

Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

edited 51 mins ago

user2313067

2,1001911

edited 51 mins ago

user2313067

2,1001911

answered 5 hours ago

DaveyDavey

961

New contributor

Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

answered 5 hours ago

DaveyDavey

961

5

Instead of trying to tail a live file, use tee instead. It was made to do exactly what you're trying to do.

From man tee:

tee(1) - Linux man page

Name tee - read from standard input and write to standard output and files

Synopsis

tee [OPTION]... [FILE]...

Description

Copy standard input to each FILE, and also to standard output.

-a, --append 
 append to the given FILEs, do not overwrite 
-i, --ignore-interrupts 
 ignore interrupt signals 
--help 
 display this help and exit 
--version
 output version information and exit

If a FILE is -, copy again to standard output.

So in your case you'd run:

python myprog.py | tee output.txt

share|improve this answer

answered 5 hours ago

n8ten8te

5,19272234

add a comment | 

5

Instead of trying to tail a live file, use tee instead. It was made to do exactly what you're trying to do.

From man tee:

tee(1) - Linux man page

Name tee - read from standard input and write to standard output and files

Synopsis

tee [OPTION]... [FILE]...

Description

Copy standard input to each FILE, and also to standard output.

-a, --append 
 append to the given FILEs, do not overwrite 
-i, --ignore-interrupts 
 ignore interrupt signals 
--help 
 display this help and exit 
--version
 output version information and exit

If a FILE is -, copy again to standard output.

So in your case you'd run:

python myprog.py | tee output.txt

share|improve this answer

answered 5 hours ago

n8ten8te

5,19272234

add a comment | 

Instead of trying to tail a live file, use tee instead. It was made to do exactly what you're trying to do.

From man tee:

tee(1) - Linux man page

Name tee - read from standard input and write to standard output and files

Synopsis

tee [OPTION]... [FILE]...

Description

Copy standard input to each FILE, and also to standard output.

-a, --append 
 append to the given FILEs, do not overwrite 
-i, --ignore-interrupts 
 ignore interrupt signals 
--help 
 display this help and exit 
--version
 output version information and exit

If a FILE is -, copy again to standard output.

So in your case you'd run:

python myprog.py | tee output.txt

share|improve this answer

answered 5 hours ago

n8ten8te

5,19272234

python myprog.py | tee output.txt

share|improve this answer

answered 5 hours ago

n8ten8te

5,19272234

answered 5 hours ago

n8ten8te

5,19272234

answered 5 hours ago

n8ten8te

5,19272234