txt to pcap file conversion using sed and xxd
I have used this command to successfully convert a .txt file to a .pcap file.
However these were all for .txt files I obtained from the www,but when i attempted to use it for .txt files I had personally created on another program, it did create filename.pcap, but it is empty.
Does anyone have a better alternative?
Also in as much as I appreciate all the assistance here in code that achieves the result, if anyone can refer me to the wikipedia articles regarding the subject that is relevant as to why this occurs, thats really going to help me contextually understand how things work, where as just the code as helpful as I feel as it this is, will leave me none the wiser as far as how linux works. So I'm sure this will be considered a duplicate question, but I actually would really like to understand specifically this scenario, rather than be referred to another apt package.
Many thanks
sed 's/^[0-9:]*//' filename.txt | sed 's/^ //g' | sed 's/ .*$//g' | xxd -r -p > filename.pcap
Updated Edit:
The only consistent difference between the collection of txt files for which the above worked, and those I created, was that mine exclusively consist of numbers, one per line, 10 digits in length, where as the others contain everything else on the keyboard.
Don't know if that's any help, and yeah also I took a look at a pcap file in the text editor simply by changing it's extension to txt, and it appears they are encrypted with a cipher that utilizes many more characters beyond what is on the standard keyboard, so... yes it's pretty standard, not sure what I was expecting there.
command-line scripts sed
add a comment |
I have used this command to successfully convert a .txt file to a .pcap file.
However these were all for .txt files I obtained from the www,but when i attempted to use it for .txt files I had personally created on another program, it did create filename.pcap, but it is empty.
Does anyone have a better alternative?
Also in as much as I appreciate all the assistance here in code that achieves the result, if anyone can refer me to the wikipedia articles regarding the subject that is relevant as to why this occurs, thats really going to help me contextually understand how things work, where as just the code as helpful as I feel as it this is, will leave me none the wiser as far as how linux works. So I'm sure this will be considered a duplicate question, but I actually would really like to understand specifically this scenario, rather than be referred to another apt package.
Many thanks
sed 's/^[0-9:]*//' filename.txt | sed 's/^ //g' | sed 's/ .*$//g' | xxd -r -p > filename.pcap
Updated Edit:
The only consistent difference between the collection of txt files for which the above worked, and those I created, was that mine exclusively consist of numbers, one per line, 10 digits in length, where as the others contain everything else on the keyboard.
Don't know if that's any help, and yeah also I took a look at a pcap file in the text editor simply by changing it's extension to txt, and it appears they are encrypted with a cipher that utilizes many more characters beyond what is on the standard keyboard, so... yes it's pretty standard, not sure what I was expecting there.
command-line scripts sed
IMHO it's going to be hard to answer this satisfactorily without examples of files that it does work for and files that it doesn't - basically thesed
command deletes (well, substitutes with nothing) various text strings: whether that results in something that will convert to a valid PCAP file rather depends on the original contents offilename.txt
– steeldriver
Feb 14 at 16:07
... also FYI theg
modifiers make little sense when applied to anchored expressions such as these (a pattern can only have one start and one end)
– steeldriver
Feb 14 at 16:10
@steeldriver thanks well most of the ones I obtained from the www were considerably larger than mine, and opening them in the text editor was just too much for my little laptop! But I have found a few that are able to be opened now, and looking at the first of them., the first thing that is different to mine is a "header" that is as follows: !@#$% !@#$%^ !@#$%^& !@#$%^&* *
– Adam
Feb 14 at 16:20
fyi, white space there is meant to represent a new line, but it won't let me do that on SE comments or at least the regular way isn't working
– Adam
Feb 14 at 16:24
1
You can edit your question to add (properly formatted) text - comments aren't a good place for that, as you've discovered
– steeldriver
Feb 14 at 16:38
add a comment |
I have used this command to successfully convert a .txt file to a .pcap file.
However these were all for .txt files I obtained from the www,but when i attempted to use it for .txt files I had personally created on another program, it did create filename.pcap, but it is empty.
Does anyone have a better alternative?
Also in as much as I appreciate all the assistance here in code that achieves the result, if anyone can refer me to the wikipedia articles regarding the subject that is relevant as to why this occurs, thats really going to help me contextually understand how things work, where as just the code as helpful as I feel as it this is, will leave me none the wiser as far as how linux works. So I'm sure this will be considered a duplicate question, but I actually would really like to understand specifically this scenario, rather than be referred to another apt package.
Many thanks
sed 's/^[0-9:]*//' filename.txt | sed 's/^ //g' | sed 's/ .*$//g' | xxd -r -p > filename.pcap
Updated Edit:
The only consistent difference between the collection of txt files for which the above worked, and those I created, was that mine exclusively consist of numbers, one per line, 10 digits in length, where as the others contain everything else on the keyboard.
Don't know if that's any help, and yeah also I took a look at a pcap file in the text editor simply by changing it's extension to txt, and it appears they are encrypted with a cipher that utilizes many more characters beyond what is on the standard keyboard, so... yes it's pretty standard, not sure what I was expecting there.
command-line scripts sed
I have used this command to successfully convert a .txt file to a .pcap file.
However these were all for .txt files I obtained from the www,but when i attempted to use it for .txt files I had personally created on another program, it did create filename.pcap, but it is empty.
Does anyone have a better alternative?
Also in as much as I appreciate all the assistance here in code that achieves the result, if anyone can refer me to the wikipedia articles regarding the subject that is relevant as to why this occurs, thats really going to help me contextually understand how things work, where as just the code as helpful as I feel as it this is, will leave me none the wiser as far as how linux works. So I'm sure this will be considered a duplicate question, but I actually would really like to understand specifically this scenario, rather than be referred to another apt package.
Many thanks
sed 's/^[0-9:]*//' filename.txt | sed 's/^ //g' | sed 's/ .*$//g' | xxd -r -p > filename.pcap
Updated Edit:
The only consistent difference between the collection of txt files for which the above worked, and those I created, was that mine exclusively consist of numbers, one per line, 10 digits in length, where as the others contain everything else on the keyboard.
Don't know if that's any help, and yeah also I took a look at a pcap file in the text editor simply by changing it's extension to txt, and it appears they are encrypted with a cipher that utilizes many more characters beyond what is on the standard keyboard, so... yes it's pretty standard, not sure what I was expecting there.
command-line scripts sed
command-line scripts sed
edited Feb 14 at 17:16
Adam
asked Feb 14 at 15:44
AdamAdam
1337
1337
IMHO it's going to be hard to answer this satisfactorily without examples of files that it does work for and files that it doesn't - basically thesed
command deletes (well, substitutes with nothing) various text strings: whether that results in something that will convert to a valid PCAP file rather depends on the original contents offilename.txt
– steeldriver
Feb 14 at 16:07
... also FYI theg
modifiers make little sense when applied to anchored expressions such as these (a pattern can only have one start and one end)
– steeldriver
Feb 14 at 16:10
@steeldriver thanks well most of the ones I obtained from the www were considerably larger than mine, and opening them in the text editor was just too much for my little laptop! But I have found a few that are able to be opened now, and looking at the first of them., the first thing that is different to mine is a "header" that is as follows: !@#$% !@#$%^ !@#$%^& !@#$%^&* *
– Adam
Feb 14 at 16:20
fyi, white space there is meant to represent a new line, but it won't let me do that on SE comments or at least the regular way isn't working
– Adam
Feb 14 at 16:24
1
You can edit your question to add (properly formatted) text - comments aren't a good place for that, as you've discovered
– steeldriver
Feb 14 at 16:38
add a comment |
IMHO it's going to be hard to answer this satisfactorily without examples of files that it does work for and files that it doesn't - basically thesed
command deletes (well, substitutes with nothing) various text strings: whether that results in something that will convert to a valid PCAP file rather depends on the original contents offilename.txt
– steeldriver
Feb 14 at 16:07
... also FYI theg
modifiers make little sense when applied to anchored expressions such as these (a pattern can only have one start and one end)
– steeldriver
Feb 14 at 16:10
@steeldriver thanks well most of the ones I obtained from the www were considerably larger than mine, and opening them in the text editor was just too much for my little laptop! But I have found a few that are able to be opened now, and looking at the first of them., the first thing that is different to mine is a "header" that is as follows: !@#$% !@#$%^ !@#$%^& !@#$%^&* *
– Adam
Feb 14 at 16:20
fyi, white space there is meant to represent a new line, but it won't let me do that on SE comments or at least the regular way isn't working
– Adam
Feb 14 at 16:24
1
You can edit your question to add (properly formatted) text - comments aren't a good place for that, as you've discovered
– steeldriver
Feb 14 at 16:38
IMHO it's going to be hard to answer this satisfactorily without examples of files that it does work for and files that it doesn't - basically the
sed
command deletes (well, substitutes with nothing) various text strings: whether that results in something that will convert to a valid PCAP file rather depends on the original contents of filename.txt
– steeldriver
Feb 14 at 16:07
IMHO it's going to be hard to answer this satisfactorily without examples of files that it does work for and files that it doesn't - basically the
sed
command deletes (well, substitutes with nothing) various text strings: whether that results in something that will convert to a valid PCAP file rather depends on the original contents of filename.txt
– steeldriver
Feb 14 at 16:07
... also FYI the
g
modifiers make little sense when applied to anchored expressions such as these (a pattern can only have one start and one end)– steeldriver
Feb 14 at 16:10
... also FYI the
g
modifiers make little sense when applied to anchored expressions such as these (a pattern can only have one start and one end)– steeldriver
Feb 14 at 16:10
@steeldriver thanks well most of the ones I obtained from the www were considerably larger than mine, and opening them in the text editor was just too much for my little laptop! But I have found a few that are able to be opened now, and looking at the first of them., the first thing that is different to mine is a "header" that is as follows: !@#$% !@#$%^ !@#$%^& !@#$%^&* *
– Adam
Feb 14 at 16:20
@steeldriver thanks well most of the ones I obtained from the www were considerably larger than mine, and opening them in the text editor was just too much for my little laptop! But I have found a few that are able to be opened now, and looking at the first of them., the first thing that is different to mine is a "header" that is as follows: !@#$% !@#$%^ !@#$%^& !@#$%^&* *
– Adam
Feb 14 at 16:20
fyi, white space there is meant to represent a new line, but it won't let me do that on SE comments or at least the regular way isn't working
– Adam
Feb 14 at 16:24
fyi, white space there is meant to represent a new line, but it won't let me do that on SE comments or at least the regular way isn't working
– Adam
Feb 14 at 16:24
1
1
You can edit your question to add (properly formatted) text - comments aren't a good place for that, as you've discovered
– steeldriver
Feb 14 at 16:38
You can edit your question to add (properly formatted) text - comments aren't a good place for that, as you've discovered
– steeldriver
Feb 14 at 16:38
add a comment |
2 Answers
2
active
oldest
votes
You mention that
The only consistent difference between the collection of txt files for
which the above worked, and those I created, was that mine exclusively
consist of numbers, one per line, 10 digits in length
Assuming by "numbers" you mean sequences of decimal digits, then the first sed
expression
s/^[0-9:]*//
meaning match zero or more decimal digits or colon characters anchored to the start of the line, and replace them with nothing (which is presumably intended to remove the default byte offset from a regular xxd
output) will remove everything, leaving only a sequence of empty lines.
ok so is there a "control character" of sorts that I could use to de-anchor each digit sequence from it's respective line?
– Adam
Feb 14 at 18:52
also if this is the case I would have thought the method Charles proposed would eliminate the issue since we are not piping anything from sed to xxd input
– Adam
Feb 14 at 18:56
@Adam until you tell us what is in yourfilename.txt
file and how it relates topcap
, this is an XY problem. FYI piping isn't the issue: you will get the same result whether you pipe the results between programs or save them in intermediate files.
– steeldriver
Feb 14 at 19:12
ok true yep I misread the part about default byte offset my bad dyslexia
– Adam
Feb 14 at 19:32
But I did tell you what is in my text file, numbers, one per line, each having exactly 10 digits
– Adam
Feb 14 at 19:33
|
show 1 more comment
I don't know sed, or xxd but I would try to follow the 4 commands individually:
sed 's/^[0-9:]*//' filename.txt > step1.txt
sed 's/^ //g' step1.txt > step2.txt
sed 's/ .*$//g' step2.txt > step3.txt
xxd -r -p step3.txt > filename.pcap
Following each step, I'd find a handy sed
tutorial, and check to see if the output of the command matches what I think it should be, thus learning a little about sed
and xxd
in the process.
You may have noticed that I removed the "|" character from the command line you originally used. "|" is a pipe, indicating that the output from the command is to be used as the input for the next command. Instead of piping the data about, the command outputs are stuffed into intermediate files, which are then used as the input to the next command.
ok thankyou very much so how would I go about learning what part of my computer handles the piping of data like as with this case, and get an idea of how this process works? Like in looking at the method you have shown me I can see now that it's not a necessity, but I'd like to learn more about it for any future scenarios that may arise where it is
– Adam
Feb 14 at 16:11
1
The pipe itself is a really low-level bit of OS stuff - a Bash tutorial is a great place to start learning about Linux, and I believe that it's included in most basic Linux tutorials.
– Charles Green
Feb 14 at 16:16
1
And since Ubuntu and Linux are open source, you can always find the source code for the commands that are executed. Some of them are entirely incomprehensible to me, as they are written in languages that I don't understand (yet)
– Charles Green
Feb 14 at 16:18
Sorry @Charles Green the same outcome occurs when I follow the steps you gave, but none the less I appreciate the help, its turning out to be quite an intriguing problem who would have thought file conversion would be interesting!
– Adam
Feb 14 at 17:01
So, knowing the input to the first step, what is the output which is in step1.txt?
– Charles Green
Feb 14 at 18:56
|
show 1 more comment
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "89"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1118249%2ftxt-to-pcap-file-conversion-using-sed-and-xxd%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You mention that
The only consistent difference between the collection of txt files for
which the above worked, and those I created, was that mine exclusively
consist of numbers, one per line, 10 digits in length
Assuming by "numbers" you mean sequences of decimal digits, then the first sed
expression
s/^[0-9:]*//
meaning match zero or more decimal digits or colon characters anchored to the start of the line, and replace them with nothing (which is presumably intended to remove the default byte offset from a regular xxd
output) will remove everything, leaving only a sequence of empty lines.
ok so is there a "control character" of sorts that I could use to de-anchor each digit sequence from it's respective line?
– Adam
Feb 14 at 18:52
also if this is the case I would have thought the method Charles proposed would eliminate the issue since we are not piping anything from sed to xxd input
– Adam
Feb 14 at 18:56
@Adam until you tell us what is in yourfilename.txt
file and how it relates topcap
, this is an XY problem. FYI piping isn't the issue: you will get the same result whether you pipe the results between programs or save them in intermediate files.
– steeldriver
Feb 14 at 19:12
ok true yep I misread the part about default byte offset my bad dyslexia
– Adam
Feb 14 at 19:32
But I did tell you what is in my text file, numbers, one per line, each having exactly 10 digits
– Adam
Feb 14 at 19:33
|
show 1 more comment
You mention that
The only consistent difference between the collection of txt files for
which the above worked, and those I created, was that mine exclusively
consist of numbers, one per line, 10 digits in length
Assuming by "numbers" you mean sequences of decimal digits, then the first sed
expression
s/^[0-9:]*//
meaning match zero or more decimal digits or colon characters anchored to the start of the line, and replace them with nothing (which is presumably intended to remove the default byte offset from a regular xxd
output) will remove everything, leaving only a sequence of empty lines.
ok so is there a "control character" of sorts that I could use to de-anchor each digit sequence from it's respective line?
– Adam
Feb 14 at 18:52
also if this is the case I would have thought the method Charles proposed would eliminate the issue since we are not piping anything from sed to xxd input
– Adam
Feb 14 at 18:56
@Adam until you tell us what is in yourfilename.txt
file and how it relates topcap
, this is an XY problem. FYI piping isn't the issue: you will get the same result whether you pipe the results between programs or save them in intermediate files.
– steeldriver
Feb 14 at 19:12
ok true yep I misread the part about default byte offset my bad dyslexia
– Adam
Feb 14 at 19:32
But I did tell you what is in my text file, numbers, one per line, each having exactly 10 digits
– Adam
Feb 14 at 19:33
|
show 1 more comment
You mention that
The only consistent difference between the collection of txt files for
which the above worked, and those I created, was that mine exclusively
consist of numbers, one per line, 10 digits in length
Assuming by "numbers" you mean sequences of decimal digits, then the first sed
expression
s/^[0-9:]*//
meaning match zero or more decimal digits or colon characters anchored to the start of the line, and replace them with nothing (which is presumably intended to remove the default byte offset from a regular xxd
output) will remove everything, leaving only a sequence of empty lines.
You mention that
The only consistent difference between the collection of txt files for
which the above worked, and those I created, was that mine exclusively
consist of numbers, one per line, 10 digits in length
Assuming by "numbers" you mean sequences of decimal digits, then the first sed
expression
s/^[0-9:]*//
meaning match zero or more decimal digits or colon characters anchored to the start of the line, and replace them with nothing (which is presumably intended to remove the default byte offset from a regular xxd
output) will remove everything, leaving only a sequence of empty lines.
answered Feb 14 at 17:42
steeldriversteeldriver
68.9k11113184
68.9k11113184
ok so is there a "control character" of sorts that I could use to de-anchor each digit sequence from it's respective line?
– Adam
Feb 14 at 18:52
also if this is the case I would have thought the method Charles proposed would eliminate the issue since we are not piping anything from sed to xxd input
– Adam
Feb 14 at 18:56
@Adam until you tell us what is in yourfilename.txt
file and how it relates topcap
, this is an XY problem. FYI piping isn't the issue: you will get the same result whether you pipe the results between programs or save them in intermediate files.
– steeldriver
Feb 14 at 19:12
ok true yep I misread the part about default byte offset my bad dyslexia
– Adam
Feb 14 at 19:32
But I did tell you what is in my text file, numbers, one per line, each having exactly 10 digits
– Adam
Feb 14 at 19:33
|
show 1 more comment
ok so is there a "control character" of sorts that I could use to de-anchor each digit sequence from it's respective line?
– Adam
Feb 14 at 18:52
also if this is the case I would have thought the method Charles proposed would eliminate the issue since we are not piping anything from sed to xxd input
– Adam
Feb 14 at 18:56
@Adam until you tell us what is in yourfilename.txt
file and how it relates topcap
, this is an XY problem. FYI piping isn't the issue: you will get the same result whether you pipe the results between programs or save them in intermediate files.
– steeldriver
Feb 14 at 19:12
ok true yep I misread the part about default byte offset my bad dyslexia
– Adam
Feb 14 at 19:32
But I did tell you what is in my text file, numbers, one per line, each having exactly 10 digits
– Adam
Feb 14 at 19:33
ok so is there a "control character" of sorts that I could use to de-anchor each digit sequence from it's respective line?
– Adam
Feb 14 at 18:52
ok so is there a "control character" of sorts that I could use to de-anchor each digit sequence from it's respective line?
– Adam
Feb 14 at 18:52
also if this is the case I would have thought the method Charles proposed would eliminate the issue since we are not piping anything from sed to xxd input
– Adam
Feb 14 at 18:56
also if this is the case I would have thought the method Charles proposed would eliminate the issue since we are not piping anything from sed to xxd input
– Adam
Feb 14 at 18:56
@Adam until you tell us what is in your
filename.txt
file and how it relates to pcap
, this is an XY problem. FYI piping isn't the issue: you will get the same result whether you pipe the results between programs or save them in intermediate files.– steeldriver
Feb 14 at 19:12
@Adam until you tell us what is in your
filename.txt
file and how it relates to pcap
, this is an XY problem. FYI piping isn't the issue: you will get the same result whether you pipe the results between programs or save them in intermediate files.– steeldriver
Feb 14 at 19:12
ok true yep I misread the part about default byte offset my bad dyslexia
– Adam
Feb 14 at 19:32
ok true yep I misread the part about default byte offset my bad dyslexia
– Adam
Feb 14 at 19:32
But I did tell you what is in my text file, numbers, one per line, each having exactly 10 digits
– Adam
Feb 14 at 19:33
But I did tell you what is in my text file, numbers, one per line, each having exactly 10 digits
– Adam
Feb 14 at 19:33
|
show 1 more comment
I don't know sed, or xxd but I would try to follow the 4 commands individually:
sed 's/^[0-9:]*//' filename.txt > step1.txt
sed 's/^ //g' step1.txt > step2.txt
sed 's/ .*$//g' step2.txt > step3.txt
xxd -r -p step3.txt > filename.pcap
Following each step, I'd find a handy sed
tutorial, and check to see if the output of the command matches what I think it should be, thus learning a little about sed
and xxd
in the process.
You may have noticed that I removed the "|" character from the command line you originally used. "|" is a pipe, indicating that the output from the command is to be used as the input for the next command. Instead of piping the data about, the command outputs are stuffed into intermediate files, which are then used as the input to the next command.
ok thankyou very much so how would I go about learning what part of my computer handles the piping of data like as with this case, and get an idea of how this process works? Like in looking at the method you have shown me I can see now that it's not a necessity, but I'd like to learn more about it for any future scenarios that may arise where it is
– Adam
Feb 14 at 16:11
1
The pipe itself is a really low-level bit of OS stuff - a Bash tutorial is a great place to start learning about Linux, and I believe that it's included in most basic Linux tutorials.
– Charles Green
Feb 14 at 16:16
1
And since Ubuntu and Linux are open source, you can always find the source code for the commands that are executed. Some of them are entirely incomprehensible to me, as they are written in languages that I don't understand (yet)
– Charles Green
Feb 14 at 16:18
Sorry @Charles Green the same outcome occurs when I follow the steps you gave, but none the less I appreciate the help, its turning out to be quite an intriguing problem who would have thought file conversion would be interesting!
– Adam
Feb 14 at 17:01
So, knowing the input to the first step, what is the output which is in step1.txt?
– Charles Green
Feb 14 at 18:56
|
show 1 more comment
I don't know sed, or xxd but I would try to follow the 4 commands individually:
sed 's/^[0-9:]*//' filename.txt > step1.txt
sed 's/^ //g' step1.txt > step2.txt
sed 's/ .*$//g' step2.txt > step3.txt
xxd -r -p step3.txt > filename.pcap
Following each step, I'd find a handy sed
tutorial, and check to see if the output of the command matches what I think it should be, thus learning a little about sed
and xxd
in the process.
You may have noticed that I removed the "|" character from the command line you originally used. "|" is a pipe, indicating that the output from the command is to be used as the input for the next command. Instead of piping the data about, the command outputs are stuffed into intermediate files, which are then used as the input to the next command.
ok thankyou very much so how would I go about learning what part of my computer handles the piping of data like as with this case, and get an idea of how this process works? Like in looking at the method you have shown me I can see now that it's not a necessity, but I'd like to learn more about it for any future scenarios that may arise where it is
– Adam
Feb 14 at 16:11
1
The pipe itself is a really low-level bit of OS stuff - a Bash tutorial is a great place to start learning about Linux, and I believe that it's included in most basic Linux tutorials.
– Charles Green
Feb 14 at 16:16
1
And since Ubuntu and Linux are open source, you can always find the source code for the commands that are executed. Some of them are entirely incomprehensible to me, as they are written in languages that I don't understand (yet)
– Charles Green
Feb 14 at 16:18
Sorry @Charles Green the same outcome occurs when I follow the steps you gave, but none the less I appreciate the help, its turning out to be quite an intriguing problem who would have thought file conversion would be interesting!
– Adam
Feb 14 at 17:01
So, knowing the input to the first step, what is the output which is in step1.txt?
– Charles Green
Feb 14 at 18:56
|
show 1 more comment
I don't know sed, or xxd but I would try to follow the 4 commands individually:
sed 's/^[0-9:]*//' filename.txt > step1.txt
sed 's/^ //g' step1.txt > step2.txt
sed 's/ .*$//g' step2.txt > step3.txt
xxd -r -p step3.txt > filename.pcap
Following each step, I'd find a handy sed
tutorial, and check to see if the output of the command matches what I think it should be, thus learning a little about sed
and xxd
in the process.
You may have noticed that I removed the "|" character from the command line you originally used. "|" is a pipe, indicating that the output from the command is to be used as the input for the next command. Instead of piping the data about, the command outputs are stuffed into intermediate files, which are then used as the input to the next command.
I don't know sed, or xxd but I would try to follow the 4 commands individually:
sed 's/^[0-9:]*//' filename.txt > step1.txt
sed 's/^ //g' step1.txt > step2.txt
sed 's/ .*$//g' step2.txt > step3.txt
xxd -r -p step3.txt > filename.pcap
Following each step, I'd find a handy sed
tutorial, and check to see if the output of the command matches what I think it should be, thus learning a little about sed
and xxd
in the process.
You may have noticed that I removed the "|" character from the command line you originally used. "|" is a pipe, indicating that the output from the command is to be used as the input for the next command. Instead of piping the data about, the command outputs are stuffed into intermediate files, which are then used as the input to the next command.
answered Feb 14 at 16:00
Charles GreenCharles Green
14k73859
14k73859
ok thankyou very much so how would I go about learning what part of my computer handles the piping of data like as with this case, and get an idea of how this process works? Like in looking at the method you have shown me I can see now that it's not a necessity, but I'd like to learn more about it for any future scenarios that may arise where it is
– Adam
Feb 14 at 16:11
1
The pipe itself is a really low-level bit of OS stuff - a Bash tutorial is a great place to start learning about Linux, and I believe that it's included in most basic Linux tutorials.
– Charles Green
Feb 14 at 16:16
1
And since Ubuntu and Linux are open source, you can always find the source code for the commands that are executed. Some of them are entirely incomprehensible to me, as they are written in languages that I don't understand (yet)
– Charles Green
Feb 14 at 16:18
Sorry @Charles Green the same outcome occurs when I follow the steps you gave, but none the less I appreciate the help, its turning out to be quite an intriguing problem who would have thought file conversion would be interesting!
– Adam
Feb 14 at 17:01
So, knowing the input to the first step, what is the output which is in step1.txt?
– Charles Green
Feb 14 at 18:56
|
show 1 more comment
ok thankyou very much so how would I go about learning what part of my computer handles the piping of data like as with this case, and get an idea of how this process works? Like in looking at the method you have shown me I can see now that it's not a necessity, but I'd like to learn more about it for any future scenarios that may arise where it is
– Adam
Feb 14 at 16:11
1
The pipe itself is a really low-level bit of OS stuff - a Bash tutorial is a great place to start learning about Linux, and I believe that it's included in most basic Linux tutorials.
– Charles Green
Feb 14 at 16:16
1
And since Ubuntu and Linux are open source, you can always find the source code for the commands that are executed. Some of them are entirely incomprehensible to me, as they are written in languages that I don't understand (yet)
– Charles Green
Feb 14 at 16:18
Sorry @Charles Green the same outcome occurs when I follow the steps you gave, but none the less I appreciate the help, its turning out to be quite an intriguing problem who would have thought file conversion would be interesting!
– Adam
Feb 14 at 17:01
So, knowing the input to the first step, what is the output which is in step1.txt?
– Charles Green
Feb 14 at 18:56
ok thankyou very much so how would I go about learning what part of my computer handles the piping of data like as with this case, and get an idea of how this process works? Like in looking at the method you have shown me I can see now that it's not a necessity, but I'd like to learn more about it for any future scenarios that may arise where it is
– Adam
Feb 14 at 16:11
ok thankyou very much so how would I go about learning what part of my computer handles the piping of data like as with this case, and get an idea of how this process works? Like in looking at the method you have shown me I can see now that it's not a necessity, but I'd like to learn more about it for any future scenarios that may arise where it is
– Adam
Feb 14 at 16:11
1
1
The pipe itself is a really low-level bit of OS stuff - a Bash tutorial is a great place to start learning about Linux, and I believe that it's included in most basic Linux tutorials.
– Charles Green
Feb 14 at 16:16
The pipe itself is a really low-level bit of OS stuff - a Bash tutorial is a great place to start learning about Linux, and I believe that it's included in most basic Linux tutorials.
– Charles Green
Feb 14 at 16:16
1
1
And since Ubuntu and Linux are open source, you can always find the source code for the commands that are executed. Some of them are entirely incomprehensible to me, as they are written in languages that I don't understand (yet)
– Charles Green
Feb 14 at 16:18
And since Ubuntu and Linux are open source, you can always find the source code for the commands that are executed. Some of them are entirely incomprehensible to me, as they are written in languages that I don't understand (yet)
– Charles Green
Feb 14 at 16:18
Sorry @Charles Green the same outcome occurs when I follow the steps you gave, but none the less I appreciate the help, its turning out to be quite an intriguing problem who would have thought file conversion would be interesting!
– Adam
Feb 14 at 17:01
Sorry @Charles Green the same outcome occurs when I follow the steps you gave, but none the less I appreciate the help, its turning out to be quite an intriguing problem who would have thought file conversion would be interesting!
– Adam
Feb 14 at 17:01
So, knowing the input to the first step, what is the output which is in step1.txt?
– Charles Green
Feb 14 at 18:56
So, knowing the input to the first step, what is the output which is in step1.txt?
– Charles Green
Feb 14 at 18:56
|
show 1 more comment
Thanks for contributing an answer to Ask Ubuntu!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1118249%2ftxt-to-pcap-file-conversion-using-sed-and-xxd%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
IMHO it's going to be hard to answer this satisfactorily without examples of files that it does work for and files that it doesn't - basically the
sed
command deletes (well, substitutes with nothing) various text strings: whether that results in something that will convert to a valid PCAP file rather depends on the original contents offilename.txt
– steeldriver
Feb 14 at 16:07
... also FYI the
g
modifiers make little sense when applied to anchored expressions such as these (a pattern can only have one start and one end)– steeldriver
Feb 14 at 16:10
@steeldriver thanks well most of the ones I obtained from the www were considerably larger than mine, and opening them in the text editor was just too much for my little laptop! But I have found a few that are able to be opened now, and looking at the first of them., the first thing that is different to mine is a "header" that is as follows: !@#$% !@#$%^ !@#$%^& !@#$%^&* *
– Adam
Feb 14 at 16:20
fyi, white space there is meant to represent a new line, but it won't let me do that on SE comments or at least the regular way isn't working
– Adam
Feb 14 at 16:24
1
You can edit your question to add (properly formatted) text - comments aren't a good place for that, as you've discovered
– steeldriver
Feb 14 at 16:38