Suppose this string:
The fox jumped over the log.
Turning into:
The fox jumped over the log.
What is the simplest (1-2 lines) to achieve this, without splitting and going into lists?
pythonregexstring
Suppose this string:
The fox jumped over the log.
Turning into:
The fox jumped over the log.
What is the simplest (1-2 lines) to achieve this, without splitting and going into lists?
On Python ≥ 3.5, use pathlib.Path.mkdir
:
from pathlib import Path
Path("/my/directory").mkdir(parents=True, exist_ok=True)
For older versions of Python, I see two answers with good qualities, each with a small flaw, so I will give my take on it:
Try os.path.exists
, and consider os.makedirs
for the creation.
import os
if not os.path.exists(directory):
os.makedirs(directory)
As noted in comments and elsewhere, there's a race condition – if the directory is created between the os.path.exists
and the os.makedirs
calls, the os.makedirs
will fail with an OSError
. Unfortunately, blanket-catching OSError
and continuing is not foolproof, as it will ignore a failure to create the directory due to other factors, such as insufficient permissions, full disk, etc.
One option would be to trap the OSError
and examine the embedded error code (see Is there a cross-platform way of getting information from Python’s OSError):
import os, errno
try:
os.makedirs(directory)
except OSError as e:
if e.errno != errno.EEXIST:
raise
Alternatively, there could be a second os.path.exists
, but suppose another created the directory after the first check, then removed it before the second one – we could still be fooled.
Depending on the application, the danger of concurrent operations may be more or less than the danger posed by other factors such as file permissions. The developer would have to know more about the particular application being developed and its expected environment before choosing an implementation.
Modern versions of Python improve this code quite a bit, both by exposing FileExistsError
(in 3.3+)...
try:
os.makedirs("path/to/directory")
except FileExistsError:
# directory already exists
pass
...and by allowing a keyword argument to os.makedirs
called exist_ok
(in 3.2+).
os.makedirs("path/to/directory", exist_ok=True) # succeeds even if directory exists.
Summarize other answers I found 11 main ways to do this (see below). And I wrote some performance tests (see results below):
Ways to convert an InputStream to a String:
Using IOUtils.toString
(Apache Utils)
String result = IOUtils.toString(inputStream, StandardCharsets.UTF_8);
Using CharStreams
(Guava)
String result = CharStreams.toString(new InputStreamReader(
inputStream, Charsets.UTF_8));
Using Scanner
(JDK)
Scanner s = new Scanner(inputStream).useDelimiter("\\A");
String result = s.hasNext() ? s.next() : "";
Using Stream API (Java 8). Warning: This solution converts different line breaks (like \r\n
) to \n
.
String result = new BufferedReader(new InputStreamReader(inputStream))
.lines().collect(Collectors.joining("\n"));
Using parallel Stream API (Java 8). Warning: This solution converts different line breaks (like \r\n
) to \n
.
String result = new BufferedReader(new InputStreamReader(inputStream))
.lines().parallel().collect(Collectors.joining("\n"));
Using InputStreamReader
and StringBuilder
(JDK)
int bufferSize = 1024;
char[] buffer = new char[bufferSize];
StringBuilder out = new StringBuilder();
Reader in = new InputStreamReader(stream, StandardCharsets.UTF_8);
for (int numRead; (numRead = in.read(buffer, 0, buffer.length)) > 0; ) {
out.append(buffer, 0, numRead);
}
return out.toString();
Using StringWriter
and IOUtils.copy
(Apache Commons)
StringWriter writer = new StringWriter();
IOUtils.copy(inputStream, writer, "UTF-8");
return writer.toString();
Using ByteArrayOutputStream
and inputStream.read
(JDK)
ByteArrayOutputStream result = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
for (int length; (length = inputStream.read(buffer)) != -1; ) {
result.write(buffer, 0, length);
}
// StandardCharsets.UTF_8.name() > JDK 7
return result.toString("UTF-8");
Using BufferedReader
(JDK). Warning: This solution converts different line breaks (like \n\r
) to line.separator
system property (for example, in Windows to "\r\n").
String newLine = System.getProperty("line.separator");
BufferedReader reader = new BufferedReader(
new InputStreamReader(inputStream));
StringBuilder result = new StringBuilder();
for (String line; (line = reader.readLine()) != null; ) {
if (result.length() > 0) {
result.append(newLine);
}
result.append(line);
}
return result.toString();
Using BufferedInputStream
and ByteArrayOutputStream
(JDK)
BufferedInputStream bis = new BufferedInputStream(inputStream);
ByteArrayOutputStream buf = new ByteArrayOutputStream();
for (int result = bis.read(); result != -1; result = bis.read()) {
buf.write((byte) result);
}
// StandardCharsets.UTF_8.name() > JDK 7
return buf.toString("UTF-8");
Using inputStream.read()
and StringBuilder
(JDK). Warning: This solution has problems with Unicode, for example with Russian text (works correctly only with non-Unicode text)
StringBuilder sb = new StringBuilder();
for (int ch; (ch = inputStream.read()) != -1; ) {
sb.append((char) ch);
}
return sb.toString();
Warning:
Solutions 4, 5 and 9 convert different line breaks to one.
Solution 11 can't work correctly with Unicode text
Performance tests
Performance tests for small String
(length = 175), url in github (mode = Average Time, system = Linux, score 1,343 is the best):
Benchmark Mode Cnt Score Error Units
8. ByteArrayOutputStream and read (JDK) avgt 10 1,343 ± 0,028 us/op
6. InputStreamReader and StringBuilder (JDK) avgt 10 6,980 ± 0,404 us/op
10. BufferedInputStream, ByteArrayOutputStream avgt 10 7,437 ± 0,735 us/op
11. InputStream.read() and StringBuilder (JDK) avgt 10 8,977 ± 0,328 us/op
7. StringWriter and IOUtils.copy (Apache) avgt 10 10,613 ± 0,599 us/op
1. IOUtils.toString (Apache Utils) avgt 10 10,605 ± 0,527 us/op
3. Scanner (JDK) avgt 10 12,083 ± 0,293 us/op
2. CharStreams (guava) avgt 10 12,999 ± 0,514 us/op
4. Stream Api (Java 8) avgt 10 15,811 ± 0,605 us/op
9. BufferedReader (JDK) avgt 10 16,038 ± 0,711 us/op
5. parallel Stream Api (Java 8) avgt 10 21,544 ± 0,583 us/op
Performance tests for big String
(length = 50100), url in github (mode = Average Time, system = Linux, score 200,715 is the best):
Benchmark Mode Cnt Score Error Units
8. ByteArrayOutputStream and read (JDK) avgt 10 200,715 ± 18,103 us/op
1. IOUtils.toString (Apache Utils) avgt 10 300,019 ± 8,751 us/op
6. InputStreamReader and StringBuilder (JDK) avgt 10 347,616 ± 130,348 us/op
7. StringWriter and IOUtils.copy (Apache) avgt 10 352,791 ± 105,337 us/op
2. CharStreams (guava) avgt 10 420,137 ± 59,877 us/op
9. BufferedReader (JDK) avgt 10 632,028 ± 17,002 us/op
5. parallel Stream Api (Java 8) avgt 10 662,999 ± 46,199 us/op
4. Stream Api (Java 8) avgt 10 701,269 ± 82,296 us/op
10. BufferedInputStream, ByteArrayOutputStream avgt 10 740,837 ± 5,613 us/op
3. Scanner (JDK) avgt 10 751,417 ± 62,026 us/op
11. InputStream.read() and StringBuilder (JDK) avgt 10 2919,350 ± 1101,942 us/op
Graphs (performance tests depending on Input Stream length in Windows 7 system)
Performance test (Average Time) depending on Input Stream length in Windows 7 system:
length 182 546 1092 3276 9828 29484 58968
test8 0.38 0.938 1.868 4.448 13.412 36.459 72.708
test4 2.362 3.609 5.573 12.769 40.74 81.415 159.864
test5 3.881 5.075 6.904 14.123 50.258 129.937 166.162
test9 2.237 3.493 5.422 11.977 45.98 89.336 177.39
test6 1.261 2.12 4.38 10.698 31.821 86.106 186.636
test7 1.601 2.391 3.646 8.367 38.196 110.221 211.016
test1 1.529 2.381 3.527 8.411 40.551 105.16 212.573
test3 3.035 3.934 8.606 20.858 61.571 118.744 235.428
test2 3.136 6.238 10.508 33.48 43.532 118.044 239.481
test10 1.593 4.736 7.527 20.557 59.856 162.907 323.147
test11 3.913 11.506 23.26 68.644 207.591 600.444 1211.545
Best Answer