Idea not clear [PC operation from the BIOS upward - gulp!]

Go To Last Post
33 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hello all,

I have been having this doubt for months, though i didn't take serious efforts in getting it clear. I found that people in this forum can easily through light on it.

See i know that Microcontrollers and microprocessors execute machine instructions, since it is difficult to write with machine instructions, assembly and all those high level languages have been developed. So, for using several different modules and protocols etc we need to write codes and compile it to get the machine instructions. In case of reusing codes some library files can be included and can be compiled to get the machine instructions as a whole (it includes the codes for library).

But when we develop some applications in windows or linux we use some API (Like black box) and compile our application to get the executable file (does not have the instructions for the API functions). So how our application gets linked to the OS functions which are already existing in the system.

I guess you have understood my question. I need to know the big picture of OS from the bottom (machine instructions)

Thanks in advance

-Krishna Balan S

-------------------------------------------------------------------------

"Heroes are ordinary people with extraordinary commitment"

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

So how our application gets linked to the OS functions which are already existing in the system.

There are two possibilites. One is called "static linking" and the other is "dynamic linking". In static linking you build your code then link against .a or .lib files and the code of routines such as sin() and printf() are built into your own .exe (or other executable on Linux). The other method involves building against a "stub" library which does not contain the actual code of the functions but knows which .ldd or .so file they are in. With this kind of program, when you run it the OS "loads" it into memory and, at the same time, loads the associated .dll or .so into memory (if they are not already there). This means that the .exe can be much smaller but it does mean that to distribute a copy you also have to provide the .dll's or .so's it relies on or otherwise ensure that the user of the code will already have them on their system. In Windows things like gdi(32).dll, user(32).dll, kernel(32).dll and so on are on everyone's system and they provide the majority of the Win32 API. In Linux things like libc.n.m.so are on everyones system so all the libc functions are accessible at run time.

In Windows you can download a hugely useful program called Dependency Walker: http://www.dependencywalker.com/ and then load any Windows .exe and it will list out all the .dlls that program relies on. In Linux the equivalent is the ldd command which outputs for example:

root@ws-czc1138h0b:/sbin# ldd ifconfig
        linux-vdso.so.1 =>  (0x00007fffdddff000)
        libc.so.6 => /lib/libc.so.6 (0x00007f2cd5168000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f2cd550c000)
root@ws-czc1138h0b:/sbin#

which shows the .so files that the "ifconfig" command is dependent on.

If you don't mind large .exe's then static linking is often the easiest option as it puts all the code in one file that is easily debuggable and distributable.

When you built programs for AVR (certainly with GCC anyway) then all linking is "static". If you use sin() it will be taken from libm.a and added to your binary. If you use printf() it will be taken from libc.a and added to your program and even if you just use "wide" versions of *, /, + and - these may even involve some library code in libgcc.a being taken from the library and added to your own binary.

It's "static" because it happens once and for all time at the moment the linker runs.

There can be no concept of dynamic linking for AVR apps because there is no operating system to "load" the right binaries that provide the library functions and, anyway, all the executable code has to be in flash anyway so there's no possible mechanism for loading bits from disk or memory card when required in the way that .dll's and .so's are loaded.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

But when we develop some applications in windows or linux we use some API (Like black box) and compile our application to get the executable file (does not have the instructions for the API functions).

This is not entirely true. There are two different linking mechanisms:

Static linking, in which case the library code is actually incorporated into your executable (so if you do two different applications using this mechanism you will have that machine code in two places, one copy in each application binary).

Dynamic linking, where there will be only one copy of the machine code. The first executable that needs it will be discovered by the run-time linking mechanism which will take care of loading the dynamic library also. The next application that starts will then be linked to the already loaded library. In windows you know these libraries as DLLs.

(In the ar-gcc/avr-libc tool chain for our embedded systems we use static linking. There will never be two applications running at once, so we will not waste space by linking statically.)

For DLLs there is normally still a small library statically linked to your application. This library contains small and simple funxtions ("stubs") that in turn do the call to the dynamically linked library.

On top of that Microsoft has built some DLL API specifications. If a DLL adheres to those specifications it can be interrogated as to what it actually contains - it can report on which functions it contains, how parameter lists and retrn values look etc. Furthermore, you have inheritance so that a certain DLL can implement an interface specification that is elsewhere. The application can be built with knowledge only of this specification. Thus you can e.g. have several suppliers delivering the same functionality in form of a DLL that adheres to the interfacwe speciication. Microsoft calls this technique COM (Component Object Model). It has been extended to support calls over a network (the app executes on one machine and the DLL library executes on a different machine - this is known as Dynamic COM or DCOM.

You still have a lot of uncovered ground, and that is some modern languages/environments like C#/VB/.NET and Java. Here there are additional levels of abstraction..

And this is only one aspect of what an operating system actually is/does.

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

With the two most used operating systems on Earth, you want to know the low level details? Make Google your friend - it's all on the interwebs.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi Cliff thanks for your good explanation.

Quote:
In static linking you build your code then link against .a or .lib files and the code of routines such as sin() and printf() are built into your own .exe (or other executable on Linux).

So in case of static linking, for example if the code for printf() is added into the binary of the application, the printf() function will in turn use some functions of OS which are required for displaying or controlling the monitor. How does these functions are called ?. Like which program or what mechanism is followed to know the handle of the required functions to be used in the calling place in the application binary.

-Krishna Balan S

-------------------------------------------------------------------------

"Heroes are ordinary people with extraordinary commitment"

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

for example if the code for printf() is added into the binary of the application, the printf() function will in turn use some functions of OS which are required for displaying or controlling the monitor. How does these functions are called ?

In the case of Windows, the Windows API is called. Generally through dynamic linking (i.e. a DLL that is part of Windows per se is called). See ablove for "dynamic linking". This will tricke down, ultimately calling a "device driver" for the specific device, which will manipulate the actual hardware.

The thing you printf could of-course be ending up in e.g. a text file. The part that formats the data into a string is part of the C runt-time (statically or dynamically linked to your app). The generic part of outputting a string somewhere is part of Windows proper. The specifics on doing this to a certain device (hard disk, serial port etc) is in the device driver.

HTH!

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
n the case of Windows, the Windows API is called. Generally through dynamic linking (i.e. a DLL that is part of Windows per se is called). See ablove for "dynamic linking". This will tricke down, ultimately calling a "device driver" for the specific device, which will manipulate the actual hardware.

So even if we use static linking or dynamic linking for our application, if we look at the lower level the OS functions are called using some dynamic linking mechanisms. Is my notion right ?

-Krishna Balan S

-------------------------------------------------------------------------

"Heroes are ordinary people with extraordinary commitment"

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
In the case of Windows, the Windows API is called. Generally through dynamic linking (i.e. a DLL that is part of Windows per se is called). See ablove for "dynamic linking". This will tricke down, ultimately calling a "device driver" for the specific device, which will manipulate the actual hardware.

what was done by microsoft before inventing DLL ?
I get to know that before inventing dll some OS .exe files (kernel.exe, user32.exe) are used by the applications through API. How was it done with exe files ?

Please bear with me for asking lot of question, though those questions wont help to do something fruitful now but a good understanding will always help in the stride !

-Krishna Balan S

-------------------------------------------------------------------------

"Heroes are ordinary people with extraordinary commitment"

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm not sure exactly how Windows or Linux for that matter handles the o/s calls. Historically there was a software interrupt (or trap) that was invoked and a function called and parameters either stored in registers or on the stack. From then on the o/s would do what was requested and then return with the results either in registers or on the stack.

With ms-dos, there was a software interrupt that worked as I described.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

at the lower level the OS functions are called using some dynamic linking mechanisms.

Some OS functions are called using dynamic linking.

Quote:

what was done by microsoft before inventing DLL ?

Static linking. And "soft interrupts" which in some aspects resemble dynamic linking as we know know it (and in others dont resemble it at all). In the Evil Old Days of MS-DOS you'd call the OS by issuing a soft interrupt after setting up a parameter block telling what you wanted the OS to do for you. Similar for going directly to the BIOS. E.g.: http://en.wikipedia.org/wiki/MS-...

My memory is going (think endgame in 2001 - A Space Odyssey... [*]) but IIRC Windows up to version 3 (i.e. the one preceding NT and W95) was an really add-on on top of MS-DOS so until 1995 this was what was used. In 1995 the DLL technology was invented and in use.

---

[*] Dave Bowman is floating above/behind my head, using a special tool to turn small knobs. Transparent, dimly lit, memory mudules slide out of the back of my head, very smoothly and slowly, while I talk of more and more trivial and shallow things. "My mind is going, Dave..."

Soon I will start singing/humming... "Daisy, Daisy..."

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

My God, it's [this site] full of stars!!!!!!!

--greg
Still learning, don't shout at me, educate me.
Starting the fire is easy; the hardest part is learning how to keep the flame!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

krishnabalan,

You really need to be clear about what environment you are talking about the operation of something like printf() on.

Possibly the simplest way to operate an 8086 with a video adapter, and a keyboard is MS-DOS. Ultimately (even on AVR) printf all boils down to a single output function called putc(). That just takes one character and outputs it to the "next position" (aka "cursor position") on the output device. In DOS that is done (in 8086 assembler) with:

mov ah, 2
mov dl, ASCII_char_number
int 21h

The core of MS-DOS is a file called msdos.sys and it puts a hook in the 8086 INT 21h vector location to capture any invocations of that interrupt then it uses the AH register to know which function is being requested. AH=2 means "print single character", AH=9 means "print string terminated by '$'" and so on.

The msdos.sys (which is the same for all copies of MS-DOS) may do some processing on the character such as "code page switching" (map to a different font table) and then it passes control to the other half of MS-DOS which is a file IO.sys that in turn will use "Int 10h" with sub-function 0Eh (write char in TTY mode) or sub-function 13h (write sting). The BIOS then makes a direct memory access to the (character mode) video RAM to deposit the chosen character at the chosen position where the video scanning hardware will later find it and display on the screen.

This is MS-DOS in "text mode" and is possibly the simplest of all ways to operate the PC hardware with virtually everything being done by the ROM based BIOS, IO.sys or msdos.sys.

If you now add Windows into the mix then in the very simplest form (Win 1, 2 or 3 on top of MS-DOS) what changes is that when Windows start it calls Int 10h (BIOS video interrupt) and uses function 00h (set video mode) to switch the display from text mode to a pixel mapped graphics mode. The video buffer moves in memory from 0xB80000 to 0xA00000 and now Windows takes control of writing each pixel individually using low level .sys files and generic libraries such as gdi.dll (graphics driver interface), user.dll and kernel.dll. If you start a "command prompt" (or the Win 3) equivalent then all that happens is that a region of the screen is drawn as a black rectangle with white pixels used to display the text. If a program runs in that "window" and calls printf() then it still gets as far as Int 21h function 2 but Windows will almost certainly intercept it at that point and not allow a call through to the BIOS ROM (that doesn't even know how to draw pixels) but instead it will map it to a pixel plotting routine that just happens to know how to draw ASCII characters using a font definition.

if you now move on and write a "Windows program" is doesn't even get anywhere near Int 21h. Instead printf() just maps to some kind of TextOutput() routine in Windows (probably in gdi.dll) and that handles drawing text (usually black pixels on a white background in fact) by just plotting pixel dots in the video memory buffer.

If you move up to "modern" Windows and modern video cards then everything is running in virtual memory - the video frame buffer is no longer at an absolute address and it's MUCH bigger than a 640x480 VGA buffer (or whatever) and now there really is no such thing as "int 21h" and "int 9h" as all those things are virtualised through GDTs and IDTs and so on - but at the end of the day it's just lighting up pixels in a video buffer still.

In Linux when you printf() at a command line text based system the BIOS is not used at all. Linux has a /dev/tty (and specifically /dev/console) driver that knows how to drive the text mode video buffers directly (just like the old IBM PC BIOS in fact). If you run a graphic interface such as Gnome or KDE then it's likely running on a lower layer such as QT and that is probably running on the X Windows system and, again, that knows how to program video hardware directly to show video modes and plot pixels.

So what exactly are you asking about (and why?)?

EDIT: BTW I typed "Int 21h" into Wikipedia and it said:

http://en.wikipedia.org/wiki/INT...

I then typed in "BIOS interrupt call" and it said:

http://en.wikipedia.org/wiki/BIO...

Last Edited: Thu. Jun 2, 2011 - 06:23 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

the C runt-time

A classic!! :lol:

Quote:
Some OS functions are called using dynamic linking.

BTW regarding calling system services - in Linux it's actually Int 80h that's still used for this - it's the bridge from user mode to kernel mode and uses a dispatch table to access the right function.

(there's even a http://www.int80h.org/ website!). For a list of int 80h sub functions see:

http://lxr.linux.no/#linux+v2.6....

(if you like that is the "dynamic link table" for Linux).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi Cliff and JohanEkdahl,

The explanation is awesome. I was exactly looking for this kind of information. Now i got a good picture of what is happening under the hood.

When i was asked to choose OS or OOPS(object oriented programming) in graduate school i chose OOPS rather than OS. I guess thats the mistake i have done.

Learned a lot in this thread. Thank u very much Cliff and JohanEkdahl !!!

-Krishna Balan S

-------------------------------------------------------------------------

"Heroes are ordinary people with extraordinary commitment"

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
i chose OOPS rather than OS. I guess thats the mistake
That is certainly no mistake. The (almost) entire Arduino paradigm is based around OOPS.

--greg
Still learning, don't shout at me, educate me.
Starting the fire is easy; the hardest part is learning how to keep the flame!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

As in, oops, I bought an ardiuno?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

They have humour in Hemel Hemsptead? Amazing!!

:) :) :)

--greg
Still learning, don't shout at me, educate me.
Starting the fire is easy; the hardest part is learning how to keep the flame!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Did you mean to say "you need to have a sense of humour to live in Hemel Hempstead" ? ;-)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

BTW regarding calling system services - in Linux it's actually Int 80h that's still used for this

How does that work on an ARM? :shock:

Quote:

Quote:

the C runt-time

A classic!! Laughing


LOL! I aim to please...

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

How does that work on an ARM?

The joy of this is that the source is completely open ;-)

http://lxr.linux.no/#linux+v2.6....
http://lxr.linux.no/#linux+v2.6....

Bottom line it uses SWI.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

So

Quote:
in Linux it's actually Int 80h that's still used for this

holds only for GNU/Linux on x86. Not seeing that was what puzzled me somewhat (just enough to ask you, not enough to go looking myself).

As of January 15, 2018, Site fix-up work has begun! Now do your part and report any bugs or deficiencies here

No guarantees, but if we don't report problems they won't get much of  a chance to be fixed! Details/discussions at link given just above.

 

"Some questions have no answers."[C Baird] "There comes a point where the spoon-feeding has to stop and the independent thinking has to start." [C Lawson] "There are always ways to disagree, without being disagreeable."[E Weddington] "Words represent concepts. Use the wrong words, communicate the wrong concept." [J Morin] "Persistence only goes so far if you set yourself up for failure." [Kartman]

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

When OP said

Quote:
But when we develop some applications in windows or linux

I thought the Linux in that was probably i386 Linux as opposed to ARM Linux ?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
Did you mean to say "you need to have a sense of humour to live in Hemel Hempstead" ? ;-)

Oh yes. Half a mile from my house: http://www.buncefieldinvestigati... :shock:

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I've a friend/colleague/associate who lives that close as well. He was attending my site, the day that happened. You don't happen to know of a Steve Stanley, do you?
Probably lives 'just around the corner' from you :)
He doesn't know it, but he taught me everything I know about porting C programs across various unix platforms.

--greg
Still learning, don't shout at me, educate me.
Starting the fire is easy; the hardest part is learning how to keep the flame!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Not a name I recognise, Greg.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

In a more general sense, to do a "system call" the user program will put arguments in various "known places" and issues some sort of instruction that switches the executing code from the unpriviledged "user" context to a more priviledged "operating system" context. This system context will have access (eventually) to the actual hardware, as well as access to the user's memory and such, so it can do all the "real work" that is needed.

The exact mechanism for switching context, and exactly how much is done in which context, and exactly how many levels of privs there are, varies by CPU architecture and operating system, but exception-generating instructions like software-interrupts are pretty common.
(I'm particularly fond of the first OS I was exposed to, TOPS10, which had an "un-implemented op-code" trap, and used a bunch of the un-implemented op-codes to implement the system calls. And a bunch more so the user could write their own system-call-like functions. Unimplemented User Operations, or UUOs. As in "All a hacker needs is a tight pushj, a loose set of UUOs, and a warm place to shift.") (Yeah, showing my age.)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

(I'm particularly fond of the first OS I was exposed to, TOPS10, which had an "un-implemented op-code" trap, and used a bunch of the un-implemented op-codes to implement the system calls.

One of the first bits of code I was ever exposed to was the Sinclair Spectrum ROM code. It used the Z80 RST instructions to implement the 6 most used firmware functions - I remember thinking it was really clever at the time!

Spectrum ROM listing

See "L37B5:" for example where sin() is implemented.

Last Edited: Wed. Jun 8, 2011 - 10:05 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
One of the first bits of code I was ever exposed to was
Assembler for an ICL System 4 MainFrame. I remember thinking... 'Im never going to get a job as a software engineer!'
Ah well...

--greg
Still learning, don't shout at me, educate me.
Starting the fire is easy; the hardest part is learning how to keep the flame!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
One of the first bits of code I was ever exposed to was the Sinclair Spectrum ROM code. It used the Z80 RST instructions to implement the 6 most used firmware functions - I remember thinking it was really clever at the time!

Actually, the ancient NASCOM firmware NASSYS used RSTs as well, but drove this idea a bit further in that the RST call was followed with a one-byte system call number which the RST interrupt code used to decide which function to call. So you needed only one RST vector to have many dozens of system calls, which were in fact extendable by user functions! All that in 1982, or so! O Man, i *have* to visit my old, dusty NASCOM out there in the cold dark garage right now...

Einstein was right: "Two things are unlimited: the universe and the human stupidity. But i'm not quite sure about the former..."

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I remember when I went to university in 1981 and one of the "new" chips they introduced us to was the 68000 with it's new fangled "SWI"s - I guess they just stole the idea from Z80 RST's after all! Can't remember if 8080 had them before it? (I used to write a lot of Z80 assembler using an 8080 assembler because on CP/M, even when it ran on Z80, the supplied assembler was 8080 in fact - I wrote magazine articles for "Amstrad Computer User" (the magazine our company launched to support our home computers) and actually directed the readers to write 8080 knowing that they all had a copy of that assembler, even though every 8 bit computer we ever made were Z80 based.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You used to work for the great AMS yourself? Did you ever meet him?

--greg
Still learning, don't shout at me, educate me.
Starting the fire is easy; the hardest part is learning how to keep the flame!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

You used to work for the great AMS yourself? Did you ever meet him?

Oh don't. Sad to say but I was the original Apprentice. I joined Amstrad when I was 21 and worked for Alan for 26 years so, yes, I know him very well. He's actually a really nice bloke and not like the character he portrays on television - for example when I was seriously ill he got me to see his private doctor then arranged for hospital treatment for me in Harley Street. (but he has been known to swear and growl a bit!)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Sweet. So you are THE Cliff Lawson of Lawson Emulation ;)

--greg
Still learning, don't shout at me, educate me.
Starting the fire is easy; the hardest part is learning how to keep the flame!