How to move files to hdfs in hadoop using unix command
I have 2 directories "dft" and "hdfs" in the unpacked Hadoop folder. I am trying to copy the file(StartUnit.txt) in dft into hdfs.
The corresponding command and exception are as shown below
313159@PC213710 ~/hadoop-0.19.1
$ bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/hdfs
D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt
HADOOP_NAMENODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_SECONDARYNAMENODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_DATANODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_BALANCER_OPTS=-Dcom.sun.management.jmxremote
HADOOP_JOBTRACKER_OPTS=-Dcom.sun.management.jmxremote
HADOOP_PID_DIR=-Dcom.sun.management.jmxremote
copyFromLocal: Pathname /D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt
from D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt is not a
valid DFS filename.
Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
can someone suggest me...
unix hadoop cygwin
asked Oct 23 '13 at 10:26
Shreyuth
1111
migrated from stackoverflow.com Oct 23 '13 at 14:38
This question came from our site for professional and enthusiast programmers.
add a comment |
I have 2 directories "dft" and "hdfs" in the unpacked Hadoop folder. I am trying to copy the file(StartUnit.txt) in dft into hdfs.
The corresponding command and exception are as shown below
313159@PC213710 ~/hadoop-0.19.1
$ bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/hdfs
D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt
HADOOP_NAMENODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_SECONDARYNAMENODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_DATANODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_BALANCER_OPTS=-Dcom.sun.management.jmxremote
HADOOP_JOBTRACKER_OPTS=-Dcom.sun.management.jmxremote
HADOOP_PID_DIR=-Dcom.sun.management.jmxremote
copyFromLocal: Pathname /D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt
from D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt is not a
valid DFS filename.
Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
can someone suggest me...
unix hadoop cygwin
asked Oct 23 '13 at 10:26
Shreyuth
1111
migrated from stackoverflow.com Oct 23 '13 at 14:38
This question came from our site for professional and enthusiast programmers.
add a comment |
I have 2 directories "dft" and "hdfs" in the unpacked Hadoop folder. I am trying to copy the file(StartUnit.txt) in dft into hdfs.
The corresponding command and exception are as shown below
313159@PC213710 ~/hadoop-0.19.1
$ bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/hdfs
D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt
HADOOP_NAMENODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_SECONDARYNAMENODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_DATANODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_BALANCER_OPTS=-Dcom.sun.management.jmxremote
HADOOP_JOBTRACKER_OPTS=-Dcom.sun.management.jmxremote
HADOOP_PID_DIR=-Dcom.sun.management.jmxremote
copyFromLocal: Pathname /D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt
from D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt is not a
valid DFS filename.
Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
can someone suggest me...
unix hadoop cygwin
asked Oct 23 '13 at 10:26
Shreyuth
1111
I have 2 directories "dft" and "hdfs" in the unpacked Hadoop folder. I am trying to copy the file(StartUnit.txt) in dft into hdfs.
The corresponding command and exception are as shown below
313159@PC213710 ~/hadoop-0.19.1
$ bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/hdfs
D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt
HADOOP_NAMENODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_SECONDARYNAMENODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_DATANODE_OPTS=-Dcom.sun.management.jmxremote
HADOOP_BALANCER_OPTS=-Dcom.sun.management.jmxremote
HADOOP_JOBTRACKER_OPTS=-Dcom.sun.management.jmxremote
HADOOP_PID_DIR=-Dcom.sun.management.jmxremote
copyFromLocal: Pathname /D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt
from D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt is not a
valid DFS filename.
Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
can someone suggest me...
unix hadoop cygwin
unix hadoop cygwin
asked Oct 23 '13 at 10:26
Shreyuth
1111
asked Oct 23 '13 at 10:26
Shreyuth
1111
asked Oct 23 '13 at 10:26
Shreyuth
1111
asked Oct 23 '13 at 10:26
Shreyuth
1111
asked Oct 23 '13 at 10:26
Shreyuth
1111
1111
migrated from stackoverflow.com Oct 23 '13 at 14:38
This question came from our site for professional and enthusiast programmers.
migrated from stackoverflow.com Oct 23 '13 at 14:38
This question came from our site for professional and enthusiast programmers.
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
First the source (local file system) and then the target (HDFS) have to be specified. Run the command as below and it should work.
bin/hadoop dfs -mkdir /hdfs
bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt /hdfs/StartUnit.txt
answered Oct 23 '13 at 12:51
Praveen Sripati
58031338
hadoop fs -moveFromLocal /home/Desktop/project/data/access-0.log hdfs://localhost:54310 project/data/access-0.lognothing happen?
– NIMISHAN
Jan 4 '16 at 7:27
add a comment |
The issue is solved now. Here is the solution.
313159@PC213710 ~/hadoop-0.19.1
$ bin/hadoop fs –copyFromLocal testdriver/sample.txt /user/cts/313159/In
This command copies sample.txt file in the directory testdriver into the hdfs dir In.
If the path /user/cts/313159/In, this command creats path and copies the file into it.
edited Sep 9 '15 at 3:00
JakeGould
30.9k1093137
answered Nov 13 '13 at 5:11
Shreyuth
1111
If this is the solution to your issue, you should marke the answer as accepted.
– simlev
Mar 28 '17 at 13:00
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f664261%2fhow-to-move-files-to-hdfs-in-hadoop-using-unix-command%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
First the source (local file system) and then the target (HDFS) have to be specified. Run the command as below and it should work.
bin/hadoop dfs -mkdir /hdfs
bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt /hdfs/StartUnit.txt
answered Oct 23 '13 at 12:51
Praveen Sripati
58031338
hadoop fs -moveFromLocal /home/Desktop/project/data/access-0.log hdfs://localhost:54310 project/data/access-0.lognothing happen?
– NIMISHAN
Jan 4 '16 at 7:27
add a comment |
First the source (local file system) and then the target (HDFS) have to be specified. Run the command as below and it should work.
bin/hadoop dfs -mkdir /hdfs
bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt /hdfs/StartUnit.txt
answered Oct 23 '13 at 12:51
Praveen Sripati
58031338
hadoop fs -moveFromLocal /home/Desktop/project/data/access-0.log hdfs://localhost:54310 project/data/access-0.lognothing happen?
– NIMISHAN
Jan 4 '16 at 7:27
add a comment |
First the source (local file system) and then the target (HDFS) have to be specified. Run the command as below and it should work.
bin/hadoop dfs -mkdir /hdfs
bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt /hdfs/StartUnit.txt
answered Oct 23 '13 at 12:51
Praveen Sripati
58031338
First the source (local file system) and then the target (HDFS) have to be specified. Run the command as below and it should work.
bin/hadoop dfs -mkdir /hdfs
bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt /hdfs/StartUnit.txt
answered Oct 23 '13 at 12:51
Praveen Sripati
58031338
answered Oct 23 '13 at 12:51
Praveen Sripati
58031338
answered Oct 23 '13 at 12:51
Praveen Sripati
58031338
answered Oct 23 '13 at 12:51
Praveen Sripati
58031338
58031338
hadoop fs -moveFromLocal /home/Desktop/project/data/access-0.log hdfs://localhost:54310 project/data/access-0.lognothing happen?
– NIMISHAN
Jan 4 '16 at 7:27
add a comment |
hadoop fs -moveFromLocal /home/Desktop/project/data/access-0.log hdfs://localhost:54310 project/data/access-0.lognothing happen?
– NIMISHAN
Jan 4 '16 at 7:27
hadoop fs -moveFromLocal /home/Desktop/project/data/access-0.log hdfs://localhost:54310 project/data/access-0.log nothing happen?– NIMISHAN
Jan 4 '16 at 7:27
hadoop fs -moveFromLocal /home/Desktop/project/data/access-0.log hdfs://localhost:54310 project/data/access-0.log nothing happen?– NIMISHAN
Jan 4 '16 at 7:27
add a comment |
The issue is solved now. Here is the solution.
313159@PC213710 ~/hadoop-0.19.1
$ bin/hadoop fs –copyFromLocal testdriver/sample.txt /user/cts/313159/In
This command copies sample.txt file in the directory testdriver into the hdfs dir In.
If the path /user/cts/313159/In, this command creats path and copies the file into it.
edited Sep 9 '15 at 3:00
JakeGould
30.9k1093137
answered Nov 13 '13 at 5:11
Shreyuth
1111
If this is the solution to your issue, you should marke the answer as accepted.
– simlev
Mar 28 '17 at 13:00
add a comment |
The issue is solved now. Here is the solution.
313159@PC213710 ~/hadoop-0.19.1
$ bin/hadoop fs –copyFromLocal testdriver/sample.txt /user/cts/313159/In
This command copies sample.txt file in the directory testdriver into the hdfs dir In.
If the path /user/cts/313159/In, this command creats path and copies the file into it.
edited Sep 9 '15 at 3:00
JakeGould
30.9k1093137
answered Nov 13 '13 at 5:11
Shreyuth
1111
If this is the solution to your issue, you should marke the answer as accepted.
– simlev
Mar 28 '17 at 13:00
add a comment |
The issue is solved now. Here is the solution.
313159@PC213710 ~/hadoop-0.19.1
$ bin/hadoop fs –copyFromLocal testdriver/sample.txt /user/cts/313159/In
This command copies sample.txt file in the directory testdriver into the hdfs dir In.
If the path /user/cts/313159/In, this command creats path and copies the file into it.
edited Sep 9 '15 at 3:00
JakeGould
30.9k1093137
answered Nov 13 '13 at 5:11
Shreyuth
1111
The issue is solved now. Here is the solution.
313159@PC213710 ~/hadoop-0.19.1
$ bin/hadoop fs –copyFromLocal testdriver/sample.txt /user/cts/313159/In
This command copies sample.txt file in the directory testdriver into the hdfs dir In.
If the path /user/cts/313159/In, this command creats path and copies the file into it.
edited Sep 9 '15 at 3:00
JakeGould
30.9k1093137
answered Nov 13 '13 at 5:11
Shreyuth
1111
edited Sep 9 '15 at 3:00
JakeGould
30.9k1093137
edited Sep 9 '15 at 3:00
JakeGould
30.9k1093137
edited Sep 9 '15 at 3:00
JakeGould
30.9k1093137
30.9k1093137
answered Nov 13 '13 at 5:11
Shreyuth
1111
answered Nov 13 '13 at 5:11
Shreyuth
1111
answered Nov 13 '13 at 5:11
Shreyuth
1111
1111
If this is the solution to your issue, you should marke the answer as accepted.
– simlev
Mar 28 '17 at 13:00
add a comment |
If this is the solution to your issue, you should marke the answer as accepted.
– simlev
Mar 28 '17 at 13:00
If this is the solution to your issue, you should marke the answer as accepted.
– simlev
Mar 28 '17 at 13:00
If this is the solution to your issue, you should marke the answer as accepted.
– simlev
Mar 28 '17 at 13:00
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f664261%2fhow-to-move-files-to-hdfs-in-hadoop-using-unix-command%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
