[hercules-390] z/OS benchmark on hercules

[[e-mail] [hercules-390]]

Hey,
thanks to you I was able to get z/os 1.10 running on hercules. It uses 28 cores and I would like to run IO and MIPS Benchmarks on it. I already tried a REXX benchmark but the results were underwhelming. Does anybody now how to benchmark this properly?

[[e-mail] [hercules-390]]

Post by ***@yahoo.com [hercules-390]
I was able to get z/os 1.10 running on
hercules. It uses 28 cores and I would
like to run IO and MIPS Benchmarks
on it. I already tried a REXX benchmark
but the results were underwhelming.
Does anybody now how to benchmark
this properly?

I don't know the answer to your question,
but this recent message sprang to mind:

https://groups.yahoo.com/neo/groups/hercules-390/conversations/messages/80890

BFN. Paul.

[Joseph Reichman [e-mail] [hercules-390]]

What kind of computer do you have


I just bought a 8 core machine 3.2 - 3.7 ghz I7

64 gig of ram


Would you know if each CPU thread has a dedicated core

Post by ***@yahoo.com.au [hercules-390]

Post by ***@yahoo.com [hercules-390]
I was able to get z/os 1.10 running on
hercules. It uses 28 cores and I would
like to run IO and MIPS Benchmarks
on it. I already tried a REXX benchmark
but the results were underwhelming.
Does anybody now how to benchmark
this properly?

I don't know the answer to your question,
https://groups.yahoo.com/neo/groups/hercules-390/conversations/messages/80890
BFN. Paul.

[[e-mail] [hercules-390]]

Hey,
thank you for your answers. I just wanted to build a fairly fast hercules setup and check some numbers. So how do you guys benchmark your hercules installation? How can I upload those programs in z/os?


@Josef I'm using a server that was given to me with 2x 2683v3 14 cores and 64 gig ram.

[[e-mail] [hercules-390]]

Hey,
thank you for your answers. I just wanted to build a fairly fast hercules setup, check some numbers and play around.

@Josef I'm using a server that was given to me with 2x 2683v3 14 cores and 64 gig ram.

Like Dave said this system doesn't seem to be very suitable, which is sad because I'm just a student who wanted to emulate the best server of the world with decent performance on a "cheap" x86.

['Dave Wade' [e-mail] [hercules-390]]

I see Fish has corrected me on i/o so depending on workload you may want to emulate fewer than 28 CPU’s to allocate more resource for i/o.

It would be a great box for running a modern zOS with lots of users, but for a single job stream, I suspect my 3Ghz desktop might be faster.

The Intel docs say the CPU clock speed varies from 2ghz to 3ghz depending on core activity so although total throughput may be more with many threads, a single will run faster on a lightly loaded machine.



Dave



From: hercules-***@yahoogroups.com [mailto:hercules-***@yahoogroups.com]
Sent: 29 January 2017 15:09
To: hercules-***@yahoogroups.com
Subject: Re: [hercules-390] Re: z/OS benchmark on hercules





Hey,
thank you for your answers. I just wanted to build a fairly fast hercules setup, check some numbers and play around.

@Josef I'm using a server that was given to me with 2x 2683v3 14 cores and 64 gig ram.

Like Dave said this system doesn't seem to be very suitable, which is sad because I'm just a student who wanted to emulate the best server of the world with decent performance on a "cheap" x86.

[[e-mail] [hercules-390]]

---In hercules-***@yahoogroups.com, <***@...> wrote :

I see Fish has corrected me on i/o so depending on workload you may want to emulate fewer than 28 CPU’s to allocate more resource for i/o.
It would be a great box for running a modern zOS with lots of users, but for a single job stream, I suspect my 3Ghz desktop might be faster.
The Intel docs say the CPU clock speed varies from 2ghz to 3ghz depending on core activity so although total throughput may be more with many threads, a single will run faster on a lightly loaded machine.

Dave




Having a boatload of cpu's doesn't help all that much, as long as you have enough to cover what you need. I use a server with dual Xeon x5680's with 48GB. My homebrew benchmark says I get about 66 mips on a single cpu. Run as an 8-way, it drops to 4-5 20 mip engines with the last 2-3 at 40 mips. By comparison a real z10 (U05 I think it was) gave a benchmark of 283mips for one engine.

You can define as many cpu's as you like, that doesn't seem to make much difference. One cpu in an 8-way still gives 60 mips. It's when you start using the rest that things go south. I don't know anythng about the internals of Hercules, but my guess would be that it's controlling the memory accesses from the different cpus that kills the performance. You can get maybe three engines to run near full speed, but that's about it.

['Dave Wade' [e-mail] [hercules-390]]

Sent: 30 January 2017 04:42
Subject: RE: [hercules-390] Re: z/OS benchmark on hercules
I see Fish has corrected me on i/o so depending on workload you may want to emulate fewer than 28 CPU’s to allocate more resource for i/o.
It would be a great box for running a modern zOS with lots of users, but for a single job stream, I suspect my 3Ghz desktop might be faster.
The Intel docs say the CPU clock speed varies from 2ghz to 3ghz depending on core activity so although total throughput may be more with many threads, a single will run faster on a lightly loaded machine.
Dave
Having a boatload of cpu's doesn't help all that much, as long as you have enough to cover what you need. I use a server with dual Xeon x5680's with 48GB. My homebrew benchmark says I get about 66 mips on a single cpu. Run as an 8-way, it drops to 4-5 >20 mip engines with the last 2-3 at 40 mips. By comparison a real z10 (U05 I think it was) gave a benchmark of 283mips for one engine.
You can define as many cpu's as you like, that doesn't seem to make much difference. One cpu in an 8-way still gives 60 mips. It's when you start using the rest that things go south. I don't know anythng about the internals of Hercules, but my guess would be >that it's controlling the memory accesses from the different cpus that kills the performance. You can get maybe three engines to run near full speed, but that's about it.

That’s about what I would expect, the interlock code that ensure atomic access is very resource heavy. I know modern zOS is better at handling multiple CPU's but when I worked on Honeywell kit the rule of thumb was each CPU extra CPU added 0.8 of the last one. So

2 x cpu = 1.8,
3 x cpu = 1.8 + 0.64 = 2.44,
4 x cpu = 2.44 + . 512 = 2.95

These multi-core servers are really designed for use in a Virtual Environment where you run lots of copies of Linux or Windows on a single server. Even then CPU isn't usually the bottleneck, I have always run out of RAM before I ran out of CPU.

Dave




________________________________________
Posted by: mailto:***@sympatico.ca
________________________________________

[Ivan Warren [e-mail] [hercules-390]]

Post by ***@yahoo.com [hercules-390]
Hey,
thanks to you I was able to get z/os 1.10 running on hercules. It uses
28 cores and I would like to run IO and MIPS Benchmarks on it. I
already tried a REXX benchmark but the results were underwhelming.
Does anybody now how to benchmark this properly?

Sebastian,

Remember... Compared to an IBM z system, hercules is always going to be
slower (maybe 100x slower), because hercules is a pure software
implementation - while the IBM implementation of the architecture is
heavily hardware assisted by chips especially designed to implement the
architecture. Even IBM's pure software implementation (z/PDT) is faster
because it uses JIT emulation - something we do not have the resource to
implement - and possibly cannot legally do.

The only use the hercules community has of benchmarks is comparing
different versions modifications to hercules to check the impact.

--Ivan


[Non-text portions of this message have been removed]

[Joe Monk [e-mail] [hercules-390]]

The answer to your question is that you are trying to compare apples to
oranges.

A z/series cannot ipl os/360. Hercules can.

A z/series doesn't have the baggage that Hercules has because Hercules can
be configured as anything from a 370 to a z/series solely by changing the
configuration file. The same software thus runs, albeit in different ways.
That's why Hercules will always be slower.

Joe

Post by ***@yahoo.com [hercules-390]
Hey,
thanks to you I was able to get z/os 1.10 running on hercules. It uses
28 cores and I would like to run IO and MIPS Benchmarks on it. I
already tried a REXX benchmark but the results were underwhelming.
Does anybody now how to benchmark this properly?

Sebastian,

Remember... Compared to an IBM z system, hercules is always going to be
slower (maybe 100x slower), because hercules is a pure software
implementation - while the IBM implementation of the architecture is
heavily hardware assisted by chips especially designed to implement the
architecture. Even IBM's pure software implementation (z/PDT) is faster
because it uses JIT emulation - something we do not have the resource to
implement - and possibly cannot legally do.

The only use the hercules community has of benchmarks is comparing
different versions modifications to hercules to check the impact.

--Ivan

[Non-text portions of this message have been removed]

['Dave Wade' [e-mail] [hercules-390]]

Dear Person with no name,



Benchmarking is both an art and a science, and as with all such processes, how you do it depends on the answer you want. The problems you face , and there are many, are classical. That is how to effectively drive multiple CPU’s simultaneously. Hercules will only ever use one “Emulating” CPU at any one time in per “Emulated” CPU. MVS (or whatever) generally doesn’t support multi-threading in a single “task”. So in order to drive 28 CPU’s you are going to have to have at least 28 TSO sessions, or 28 batch jobs, or 28 CICS transactions. You don’t say how much RAM you have but I suspect that you may run out of RAM before you get 28 sessions running.



Then lets look at the I/O. I believe that Hercules only uses a single CPU to manage the i/o (I am sure someone will shout if I am wrong) which may be a major bottle neck. I am also willing to bet you don’t have 28 disks in your IO subsystem and I am willing to bet a real “Z” box will have a complex storage system with multiple fibre channel paths to a complex tiered storage area network (SAN) that optimizes data locations as the load varies. If you do have 28 disks then you may want to set things up so that one disk is used per thread, or create a huge striped set so the i/o is spread almost at random.



Then once you have done it how do you publish it? It its attributable IBM will get upset and probably demand its removal. If it is not attributable then it is of no value
..



Dave

G4UGM





From: hercules-***@yahoogroups.com [mailto:hercules-***@yahoogroups.com]
Sent: 29 January 2017 13:49
To: hercules-***@yahoogroups.com
Subject: Re: [hercules-390] z/OS benchmark on hercules








The answer to your question is that you are trying to compare apples to oranges.



A z/series cannot ipl os/360. Hercules can.



A z/series doesn't have the baggage that Hercules has because Hercules can be configured as anything from a 370 to a z/series solely by changing the configuration file. The same software thus runs, albeit in different ways. That's why Hercules will always be slower.



Joe

Post by ***@yahoo.com [hercules-390]
Hey,
thanks to you I was able to get z/os 1.10 running on hercules. It uses
28 cores and I would like to run IO and MIPS Benchmarks on it. I
already tried a REXX benchmark but the results were underwhelming.
Does anybody now how to benchmark this properly?

Sebastian,

Remember... Compared to an IBM z system, hercules is always going to be
slower (maybe 100x slower), because hercules is a pure software
implementation - while the IBM implementation of the architecture is
heavily hardware assisted by chips especially designed to implement the
architecture. Even IBM's pure software implementation (z/PDT) is faster
because it uses JIT emulation - something we do not have the resource to
implement - and possibly cannot legally do.

The only use the hercules community has of benchmarks is comparing
different versions modifications to hercules to check the impact.

--Ivan

[Non-text portions of this message have been removed]

['\'Fish\' (David B. Trout)' [e-mail] [hercules-390]]

Dave Wade wrote:

[...]

Post by 'Dave Wade' ***@gmail.com [hercules-390]
I believe that Hercules only uses a single CPU to manage
the i/o (I am sure someone will shout if I am wrong)

WRONG! (*)

https://fish-git.github.io/html/hercconf.html#DEVTMAX



(*) Was that loud enough? :)

[Ivan Warren [e-mail] [hercules-390]]

Post by '\'Fish\' (David B. Trout)' ***@gmail.com [hercules-390]
[...]

Post by 'Dave Wade' ***@gmail.com [hercules-390]
I believe that Hercules only uses a single CPU to manage
the i/o (I am sure someone will shout if I am wrong)

WRONG! (*)
https://fish-git.github.io/html/hercconf.html#DEVTMAX
(*) Was that loud enough? :)

Of course that assertion is wrong ;)

Every I/O request is dispatched to an independent thread

--Ivan



[Non-text portions of this message have been removed]

['\'Fish\' (David B. Trout)' [e-mail] [hercules-390]]

Post by Ivan Warren ***@vmfacility.fr [hercules-390]

[...]

Post by 'Dave Wade' ***@gmail.com [hercules-390]
I believe that Hercules only uses a single CPU to manage
the i/o (I am sure someone will shout if I am wrong)

WRONG! (*)
https://fish-git.github.io/html/hercconf.html#DEVTMAX
(*) Was that loud enough? :)

Of course that assertion is wrong ;)
Every I/O request is dispatched to an independent thread

And THAT assertion is *also* wrong. ;-)

It all depends on whatever 'devtmax' value you happen to choose.

But I get your point: thread != CPU. If your host only has one CPU (uniprocessor) then regardless of your devtmax value, every I/O will be managed by a single CPU. (*)


(smart aleck!)

Post by Ivan Warren ***@vmfacility.fr [hercules-390]
;-)

['Dave Wade' [e-mail] [hercules-390]]

-----Original Message-----
Sent: 29 January 2017 15:53
Subject: RE: [hercules-390] z/OS benchmark on hercules
[...]

Post by 'Dave Wade' ***@gmail.com [hercules-390]
I believe that Hercules only uses a single CPU to manage the i/o (I am
sure someone will shout if I am wrong)

WRONG! (*)
https://fish-git.github.io/html/hercconf.html#DEVTMAX

Thanks

(*) Was that loud enough? :)

Yes, disturbed my siesta

--
"Fish" (David B. Trout)
Software Development Laboratories
http://www.softdevlabs.com

Dave

[[e-mail] [hercules-390]]

Hey,
thank you for your answers. I just wanted to build a fairly fast hercules setup and check some numbers. So how do you guys benchmark your hercules installation? How can I upload those programs in z/os?

[Ivan Warren [e-mail] [hercules-390]]

Post by ***@yahoo.com [hercules-390]
Hey,
thank you for your answers. I just wanted to build a fairly fast
hercules setup and check some numbers. So how do you guys benchmark
your hercules installation? How can I upload those programs in z/os?

Frankly,

All and any benchmarking I've ever done was through standalone programs
- not anything to do with software I don't have a license for.

My benchmarks are with :
VM/370
Linux

I didn't try MVS 3.8j and only use DOS 34 to generate the VM/370 CMSDOS,
CMSBAM, CMSAMS and CMSVSAM segments using the DOSGEN (for DOS and
DOS/BAM) and VSAMGEN (FOR AMS/IDCAMS/AMSERV and VSAM) utilities.

--Ivan


[Non-text portions of this message have been removed]

[[e-mail] [hercules-390]]

Post by 'Dave Wade' ***@gmail.com [hercules-390]
The answer to your question is that you are trying to compare apples to oranges.
A z/series cannot ipl os/360. Hercules can.
A z/series doesn't have the baggage that
Hercules has because Hercules can be
configured as anything from a 370 to a
z/series solely by changing the configuration
file. The same software thus runs, albeit in
different ways. That's why Hercules will
always be slower.

Are you saying that if Hercules was modified
to strip out all the non-zArch stuff, to make
it a purely z/Arch emulator, that Hercules
would be:

1. As fast as real z/Arch hardware.
2. 1% faster than it currently is.
3. Something else?

BFN. Paul.

[Joe Monk [e-mail] [hercules-390]]

Combination of 2 and 3.

It would still be missing all of the hardware assists for the OS that IBM
includes in real hardware.

Joe

Post by Joe Monk ***@gmail.com [hercules-390]

Post by Joe Monk ***@gmail.com [hercules-390]
The answer to your question is that you are trying to compare apples to

oranges.

Post by Joe Monk ***@gmail.com [hercules-390]
A z/series cannot ipl os/360. Hercules can.
A z/series doesn't have the baggage that
Hercules has because Hercules can be
configured as anything from a 370 to a
z/series solely by changing the configuration
file. The same software thus runs, albeit in
different ways. That's why Hercules will
always be slower.

Are you saying that if Hercules was modified
to strip out all the non-zArch stuff, to make
it a purely z/Arch emulator, that Hercules
1. As fast as real z/Arch hardware.
2. 1% faster than it currently is.
3. Something else?
BFN. Paul.

[[e-mail] [hercules-390]]

'Properly' is anambiguous target. You might want to look at my '251' project showing over 250 tso users running kicks on a turnkey mvs system. I think the scripts could be easily modified to build a zos/cics simulation for whatever workload you wanted. Here's the URL kicksfortso.com/same/KooKbooK/KooKbooK-251project.htm

['Dave G4UGM' [e-mail] [hercules-390]]

Mike,

I remember you doing that. I also remember it took a lot of effort. Pretty sure it would run with a few mods on zOS.

Dave



From: hercules-***@yahoogroups.com [mailto:hercules-***@yahoogroups.com]
Sent: 29 January 2017 17:02
To: hercules-***@yahoogroups.com
Subject: [hercules-390] Re: z/OS benchmark on hercules





'Properly' is anambiguous target. You might want to look at my '251' project showing over 250 tso users running kicks on a turnkey mvs system. I think the scripts could be easily modified to build a zos/cics simulation for whatever workload you wanted. Here's the URL kicksfortso.com/same/KooKbooK/KooKbooK-251project.htm

[[e-mail] [hercules-390]]

Hello Mr Noel,
I already looked at kicks and I'm in that group too. I think that project would be intessting for me. I hope you can help me out if I'm stuck.

[[e-mail] [hercules-390]]

Hello Mr Noel,
I already looked at kicks and I'm in that group too. I think that project would be intessting for me. Maybe I can extend your demo. I hope you can help me out if I'm stuck because right now I have no idea how you did that.

[[e-mail] [hercules-390]]

Happy to help if necessary. Probably the easiest way to get going is to start with the 1st of the set (KooKbooK recipe #6, (kicksfortso.com/User's%20Guide%201.5.0/KooKbooK/06/index.htm),, cut/paste it into a turnkey mvs and see how it works, then cut/paste into zos (fixing as needed). When you get #6 running in zos move onto #7... Good luck!